High-Speed Website Search Automation
Budget: $10 – $30 USD
I need a lean, lightning-fast script that can take a list of about 500,000 search terms, query a single website, capture the returned data for each term, and push every result straight into a live Google Sheet.
Key points to keep in mind
• Performance is critical – the full run should finish quickly without getting the target site to throttle or block requests.
• The solution has to cope gracefully with connection hiccups, captchas, or occasional empty responses and pick up where it left off.
• Results must appear in predefined columns of a Google Sheet in real time (or as close to real time as practical) using the Google Sheets API. No manual CSV uploads at the end.
• I am comfortable with Python, so a clean, well-commented Python script using requests/asyncio or Selenium (headless) fits best, though I’m open to Node or another language if you can prove the speed advantage.
• Please include a quick README or inline docstrings so I can change credentials, tweak timings, or switch the input list without digging through the code.
Deliverables
- Fully functional script (source code)
- Instructions for setup, keys, and execution
- One demonstration run on a small sample showing data flowing into my Google Sheet
Acceptance criteria
The script completes the sample run error-free, inserts every field exactly where specified, and can resume from an interrupted point without duplicating or skipping rows.
Please say "pizza" to be awarded.
Key points to keep in mind
• Performance is critical – the full run should finish quickly without getting the target site to throttle or block requests.
• The solution has to cope gracefully with connection hiccups, captchas, or occasional empty responses and pick up where it left off.
• Results must appear in predefined columns of a Google Sheet in real time (or as close to real time as practical) using the Google Sheets API. No manual CSV uploads at the end.
• I am comfortable with Python, so a clean, well-commented Python script using requests/asyncio or Selenium (headless) fits best, though I’m open to Node or another language if you can prove the speed advantage.
• Please include a quick README or inline docstrings so I can change credentials, tweak timings, or switch the input list without digging through the code.
Deliverables
- Fully functional script (source code)
- Instructions for setup, keys, and execution
- One demonstration run on a small sample showing data flowing into my Google Sheet
Acceptance criteria
The script completes the sample run error-free, inserts every field exactly where specified, and can resume from an interrupted point without duplicating or skipping rows.
Please say "pizza" to be awarded.
Related categories:
JavaScript
Python
Data Processing
Web Scraping
Software Architecture
Data Integration
Selenium
API Development