Monthly E-Commerce Product Data Scrape
Budget: $750 – $1,500 USD
I need a reliable solution that pulls fresh product information from a specific website and drops it into a neat, well-structured Excel workbook each month. The data points are the standard e-commerce essentials—name, price, SKU, description, availability and any variants that appear on the page. If the site nests details behind dynamic elements, please factor that in; I still expect a complete dataset.
Your script can run in Python (BeautifulSoup, Scrapy or Selenium are all fine) or any language you prefer, as long as the final output is a tidy .xlsx file ready for analysis. I’ll trigger the run once a month, so the process should be repeatable with minimal manual tweaking—ideally a single command or scheduled task.
Acceptance criteria
• A working scraper that navigates pagination and captures every live product listing.
• Clean Excel file with separate columns for each attribute, no duplicates or missing rows.
• Clear instructions so I can rerun the job monthly on my own machine (or a lightweight hosting option you recommend).
• Basic error handling and logging so I can see what was scraped and flag pages that change or break.
Share a brief outline of your approach, mention any libraries you plan to use, and let me know if you need sample URLs before you begin.
Your script can run in Python (BeautifulSoup, Scrapy or Selenium are all fine) or any language you prefer, as long as the final output is a tidy .xlsx file ready for analysis. I’ll trigger the run once a month, so the process should be repeatable with minimal manual tweaking—ideally a single command or scheduled task.
Acceptance criteria
• A working scraper that navigates pagination and captures every live product listing.
• Clean Excel file with separate columns for each attribute, no duplicates or missing rows.
• Clear instructions so I can rerun the job monthly on my own machine (or a lightweight hosting option you recommend).
• Basic error handling and logging so I can see what was scraped and flag pages that change or break.
Share a brief outline of your approach, mention any libraries you plan to use, and let me know if you need sample URLs before you begin.
Related categories:
JavaScript
Python
Data Processing
Excel
Web Scraping
Data Mining
Scrapy
Data Extraction
BeautifulSoup
Selenium