Custom Data Enrichment Software Development
Budget: $250 – $750 USD
Im looking for a developer to build out a pretty simple straight forward scraper.
Please have existing work reflecting the requirements
Project Overview:Develop a software application that processes CSV files containing business data, enriches this data by scraping the web, and outputs the enriched data back to a CSV file. The software should be deployable on both server and local environments.
Input:
CSV file format containing the following columns:Business Address
Business Name
Business Phone Number
Output:
Enriched CSV file with additional columns:Main URL of the business
Email addresses found on the business website
Instagram business page URL
Facebook business page URL
Additional phone numbers found
Additional email addresses found
Core Functionalities:
Data Upload:Allow users to upload CSV files to the system.
Data Processing:Extract the business name and phone number from each row in the CSV.
Use the extracted information to locate the business's main website.
Web Scraping:
Scrape the main website for the following:Primary email address
Additional email addresses, if available
Additional phone numbers, if available
Identify and scrape the URLs of the business's social media pages on Instagram and Facebook.
Data Aggregation:Append the gathered data (URLs, emails, and phone numbers) to the corresponding rows in the original CSV file.
Data Export:Provide the ability to download the enriched CSV file.
Deployment:Ensure the application can be deployed on both a server and a local machine, with considerations for necessary dependencies and configurations.
Non-functional Requirements:
Ensure robust error handling for web scraping activities, especially in cases where data might not be available.
Implement rate limiting or respectful scraping practices to prevent IP banning from targeted websites.
Include logging mechanisms to capture process milestones and errors for troubleshooting.
Security Considerations:
Ensure that the upload and storage of CSV files adhere to best security practices, especially if sensitive business information is handled.
Please have existing work reflecting the requirements
Project Overview:Develop a software application that processes CSV files containing business data, enriches this data by scraping the web, and outputs the enriched data back to a CSV file. The software should be deployable on both server and local environments.
Input:
CSV file format containing the following columns:Business Address
Business Name
Business Phone Number
Output:
Enriched CSV file with additional columns:Main URL of the business
Email addresses found on the business website
Instagram business page URL
Facebook business page URL
Additional phone numbers found
Additional email addresses found
Core Functionalities:
Data Upload:Allow users to upload CSV files to the system.
Data Processing:Extract the business name and phone number from each row in the CSV.
Use the extracted information to locate the business's main website.
Web Scraping:
Scrape the main website for the following:Primary email address
Additional email addresses, if available
Additional phone numbers, if available
Identify and scrape the URLs of the business's social media pages on Instagram and Facebook.
Data Aggregation:Append the gathered data (URLs, emails, and phone numbers) to the corresponding rows in the original CSV file.
Data Export:Provide the ability to download the enriched CSV file.
Deployment:Ensure the application can be deployed on both a server and a local machine, with considerations for necessary dependencies and configurations.
Non-functional Requirements:
Ensure robust error handling for web scraping activities, especially in cases where data might not be available.
Implement rate limiting or respectful scraping practices to prevent IP banning from targeted websites.
Include logging mechanisms to capture process milestones and errors for troubleshooting.
Security Considerations:
Ensure that the upload and storage of CSV files adhere to best security practices, especially if sensitive business information is handled.
Related categories:
PHP
Business, Accounting, Human Resources & Legal
Python
Web Scraping
Software Architecture