Robust Amazon Product Scraper
Budget: $30 – $250 USD
I need a dependable, maintainable scraper that can consistently pull three key data sets from Amazon: product details (name, price, description), customer reviews with ratings, and seller information. The end goal is a clean, structured feed—CSV or JSON—that I can pipe straight into my analytics pipeline whenever required.
I’m comfortable with a Python-based stack, so feel free to lean on Scrapy, Playwright/Selenium, BeautifulSoup, or a mix that best handles Amazon’s dynamic pages, CAPTCHAs, and rate limits. Rotating proxies, user-agent rotation, and sensible back-off logic are must-haves because stability matters more to me than raw speed. Containerising the solution in Docker and making it easy to deploy on AWS (EC2, Lambda, or ECS) is strongly preferred; a cron-friendly entry point will give me the flexibility to run the scraper on any schedule later.
Please make sure pagination, variants, and edge cases (out-of-stock, price changes, review pagination) are covered. I’d also like straightforward logging so I can spot failures quickly.
Deliverables
• Fully commented source code in a Git repo
• README with setup, run, and deployment steps
• Sample config for proxy and schedule settings
• Output schema plus an example dataset pulled from a test ASIN list
• Short hand-over session or video walk-through
Acceptance criteria
• At least 95 % field coverage on the supplied test list
• Stable run on 10k products without crashing or triggering Amazon blocks
• Outputs valid, deduplicated JSON/CSV ready for import
If you have prior experience bypassing Amazon’s anti-bot systems and can show a quick proof of concept, even better. Let’s build something robust that keeps the data flowing.
I’m comfortable with a Python-based stack, so feel free to lean on Scrapy, Playwright/Selenium, BeautifulSoup, or a mix that best handles Amazon’s dynamic pages, CAPTCHAs, and rate limits. Rotating proxies, user-agent rotation, and sensible back-off logic are must-haves because stability matters more to me than raw speed. Containerising the solution in Docker and making it easy to deploy on AWS (EC2, Lambda, or ECS) is strongly preferred; a cron-friendly entry point will give me the flexibility to run the scraper on any schedule later.
Please make sure pagination, variants, and edge cases (out-of-stock, price changes, review pagination) are covered. I’d also like straightforward logging so I can spot failures quickly.
Deliverables
• Fully commented source code in a Git repo
• README with setup, run, and deployment steps
• Sample config for proxy and schedule settings
• Output schema plus an example dataset pulled from a test ASIN list
• Short hand-over session or video walk-through
Acceptance criteria
• At least 95 % field coverage on the supplied test list
• Stable run on 10k products without crashing or triggering Amazon blocks
• Outputs valid, deduplicated JSON/CSV ready for import
If you have prior experience bypassing Amazon’s anti-bot systems and can show a quick proof of concept, even better. Let’s build something robust that keeps the data flowing.