Manufacturer Site Product Data Scrape
Budget: $10 – $30 USD
I need product information collected directly from a set of manufacturer websites and delivered in a clean, structured format that I can import straight into my database or spreadsheet. The job involves building (or adapting) a scraper that navigates each site, pulls every product entry, and outputs the results in CSV or Excel alongside any associated images saved to folders with matching filenames.
The scraper must:
• Respect robots.txt and any rate-limits so we stay compliant.
• Handle pagination and variant pages automatically.
• Be written in Python—BeautifulSoup, Scrapy, or Selenium are all fine as long as the code is readable and comes with a brief README explaining setup and reruns.
Deliverables
1. Completed dataset (one row per product)
2. Image archive, if images are scraped
3. Fully documented source code + requirements.txt
I will provide the list of target manufacturer URLs—and confirm the exact fields to capture—once we start. Please let me know your estimated turnaround time and any relevant scraping projects you have tackled before so I can gauge fit quickly.
The scraper must:
• Respect robots.txt and any rate-limits so we stay compliant.
• Handle pagination and variant pages automatically.
• Be written in Python—BeautifulSoup, Scrapy, or Selenium are all fine as long as the code is readable and comes with a brief README explaining setup and reruns.
Deliverables
1. Completed dataset (one row per product)
2. Image archive, if images are scraped
3. Fully documented source code + requirements.txt
I will provide the list of target manufacturer URLs—and confirm the exact fields to capture—once we start. Please let me know your estimated turnaround time and any relevant scraping projects you have tackled before so I can gauge fit quickly.
Related categories:
Python
Web Scraping
Software Architecture
Data Mining
Scrapy
Data Extraction
BeautifulSoup
Selenium