Amazon Product Data Web Scraper
Budget: $10 – $30 USD
We are looking for an experienced Web Scraping and Automation Engineer to develop a reliable, production-ready web scraper to extract product data from Amazon search result pages.
The primary objective is to input a specific search term and extract comprehensive details for all products returned by that search, across all available result pages.
Key features:
- Dynamic Configuration: The script must not hardcode search terms. It must read the target search keyword(s) from an external configuration file (e.g., config.json, .env, or a YAML file) so we can change the target products without modifying the core logic.
- Pagination Management: The scraper must autonomously detect and navigate through all pagination pages (Next Page) for the given search term until no more products are listed.
- Anti-Bot & Detection Evasion: Amazon has strict scraping countermeasures. The script must incorporate robust anti-bot evasion techniques (e.g., human-like behavior, customized user-agents, request throttling, stealth plugins, or integration with rotating proxies/session handling) to prevent CAPTCHAs and IP bans.
- Framework Preference: The script must be developed using either Playwright or Selenium (Python or Node.js preferred) to ensure long-term maintenance, stability, and easy debugging.
- Data Output: Extracted data should be cleanly structured and exported into a standard format (CSV, Excel, or JSON).
- Data Fields to Extract (Per Product):
* Product Title / Name
* Price (Current Price if available)
* Average Rating (e.g., 4.5 out of 5 stars)
* Total Number of Reviews / Ratings Count
The primary objective is to input a specific search term and extract comprehensive details for all products returned by that search, across all available result pages.
Key features:
- Dynamic Configuration: The script must not hardcode search terms. It must read the target search keyword(s) from an external configuration file (e.g., config.json, .env, or a YAML file) so we can change the target products without modifying the core logic.
- Pagination Management: The scraper must autonomously detect and navigate through all pagination pages (Next Page) for the given search term until no more products are listed.
- Anti-Bot & Detection Evasion: Amazon has strict scraping countermeasures. The script must incorporate robust anti-bot evasion techniques (e.g., human-like behavior, customized user-agents, request throttling, stealth plugins, or integration with rotating proxies/session handling) to prevent CAPTCHAs and IP bans.
- Framework Preference: The script must be developed using either Playwright or Selenium (Python or Node.js preferred) to ensure long-term maintenance, stability, and easy debugging.
- Data Output: Extracted data should be cleanly structured and exported into a standard format (CSV, Excel, or JSON).
- Data Fields to Extract (Per Product):
* Product Title / Name
* Price (Current Price if available)
* Average Rating (e.g., 4.5 out of 5 stars)
* Total Number of Reviews / Ratings Count
Related categories:
JavaScript
Python
Web Scraping
Software Architecture
Node.js
Selenium
Automation