Daily Amazon Scraping Automation
Budget: ₹12,500 – ₹37,500 INR
I need a robust, fully automated process that pulls three specific data sets from Amazon every single day:
• Product details (titles, bullet points, specs, ASIN, images)
• Customer reviews with ratings and timestamps
• Current pricing, including promotions and historical price if visible
The solution should run on a schedule without manual intervention, gracefully handle captchas or page-layout changes, and drop the results into a clean, structured file or database table I can query—CSV or JSON is fine. Accuracy and consistency matter more than sheer speed; if a product page fails on one run, the script should log the error and retry.
I’m comfortable working with Python, so Scrapy, Requests/BeautifulSoup, or Selenium are all acceptable. If you prefer another stack that achieves the same reliability, let me know. Please include in your bid:
• A brief outline of your approach to avoiding Amazon’s anti-bot measures
• Sample output for one ASIN so I can confirm the field structure
• Estimated turnaround to get the first successful daily run in place
Once everything is stable, I’ll test the feed against my downstream pipeline before signing off.
• Product details (titles, bullet points, specs, ASIN, images)
• Customer reviews with ratings and timestamps
• Current pricing, including promotions and historical price if visible
The solution should run on a schedule without manual intervention, gracefully handle captchas or page-layout changes, and drop the results into a clean, structured file or database table I can query—CSV or JSON is fine. Accuracy and consistency matter more than sheer speed; if a product page fails on one run, the script should log the error and retry.
I’m comfortable working with Python, so Scrapy, Requests/BeautifulSoup, or Selenium are all acceptable. If you prefer another stack that achieves the same reliability, let me know. Please include in your bid:
• A brief outline of your approach to avoiding Amazon’s anti-bot measures
• Sample output for one ASIN so I can confirm the field structure
• Estimated turnaround to get the first successful daily run in place
Once everything is stable, I’ll test the feed against my downstream pipeline before signing off.
Related categories:
PHP
Python
Web Scraping
Software Architecture
Scrapy
Data Extraction
BeautifulSoup
Selenium