Python Book Data Scraping Pipeline
Budget: ₹750 – ₹1,250 INR
I have roughly ten thousand ISBN-13 codes and I need a production-ready Python pipeline that can take those codes, pull the corresponding book details from Amazon.co.jp, Amazon.com, and a small set of external APIs, then push the cleaned results straight into a Google Sheets workbook.
The pipeline must
• survive Amazon’s throttling, bot checks, and page format changes without manual babysitting,
• finish a full run on 10 k titles in a single session without crashing or silently skipping rows, and
• give me fields that are already matched and normalised so downstream staff can link them to our catalogue instantly.
Architecture is up to you: Scrapy, Playwright, headless Chrome, rotating residential proxies, Selenium, or a custom HTTP solution—whichever mix keeps the request footprint human-like and maximises up-time. What matters is that the codebase is clean, well-documented, and easy for an internal engineer to extend later.
Deliverables
1. Fully annotated Python source (PEP 8 compliant) packaged so I can run it with one command.
2. A Google Sheets connector that inserts or updates rows atomically, preserving formulas already in place.
3. README with environment setup, proxy configuration, and step-by-step deployment instructions for macOS and Ubuntu.
4. Brief test report showing a run on at least 300 sample ISBNs, including elapsed time, success rate, and any retries triggered.
Acceptance will be based on:
• ≥ 98 % scrape success on the 300-item test set,
• no Amazon “bot detected” blocks during that run, and
• correctly populated Google Sheets in the agreed format.
If you have proven experience scraping Amazon at scale and piping results into Google Sheets, I’m ready to review your plan and timeline.
The pipeline must
• survive Amazon’s throttling, bot checks, and page format changes without manual babysitting,
• finish a full run on 10 k titles in a single session without crashing or silently skipping rows, and
• give me fields that are already matched and normalised so downstream staff can link them to our catalogue instantly.
Architecture is up to you: Scrapy, Playwright, headless Chrome, rotating residential proxies, Selenium, or a custom HTTP solution—whichever mix keeps the request footprint human-like and maximises up-time. What matters is that the codebase is clean, well-documented, and easy for an internal engineer to extend later.
Deliverables
1. Fully annotated Python source (PEP 8 compliant) packaged so I can run it with one command.
2. A Google Sheets connector that inserts or updates rows atomically, preserving formulas already in place.
3. README with environment setup, proxy configuration, and step-by-step deployment instructions for macOS and Ubuntu.
4. Brief test report showing a run on at least 300 sample ISBNs, including elapsed time, success rate, and any retries triggered.
Acceptance will be based on:
• ≥ 98 % scrape success on the 300-item test set,
• no Amazon “bot detected” blocks during that run, and
• correctly populated Google Sheets in the agreed format.
If you have proven experience scraping Amazon at scale and piping results into Google Sheets, I’m ready to review your plan and timeline.
Related categories:
Python
Web Scraping
Software Architecture
Data Mining
Scrapy
Google Sheets
Selenium
API Integration