High-Performance Web Scraping Bot (Python) for Website
Budget: $250 – $750 USD
I need an experienced Python developer to rebuild and optimize a high-volume web scraping bot :
https://tgr.cl/tramites-tgr/certificado-de-movimientos-de-contribuciones/
I currently have a working script, but it uses outdated libraries and must be modernized, made faster, and made more stable.
**Project Requirements
+Scrape data for more than 500,000 input rows (Region, Comuna, Rol, Sub-Rol).
+Save all returned HTML pages into structured folders to avoid re-scraping if the process stops.
+Implement configurable parallel processing (e.g., run 2, 5, 10+ simultaneous scrapes)
+Add strong error handling, retry logic, and request throttling.
+Ensure high speed with minimal failures.
**Technical Skills Needed
Python 3.10+
Requests or HTTPX
Captchha solving APIs
lxml or BeautifulSoup
Experience with sessions, cookies, and dynamic HTML parsing
Experience with multiprocessing or asyncio for large-scale scraping
Logging and fault-tolerant architecture
**Deliverables
Fully functional scraping bot with clean, documented Python code
Config file (parallelism, API keys, output folders)
Ability to continue from where it stopped (HTML caching)
Output in CSV/XLSX
Installation/execution guide
Optional: Standalone executable
https://tgr.cl/tramites-tgr/certificado-de-movimientos-de-contribuciones/
I currently have a working script, but it uses outdated libraries and must be modernized, made faster, and made more stable.
**Project Requirements
+Scrape data for more than 500,000 input rows (Region, Comuna, Rol, Sub-Rol).
+Save all returned HTML pages into structured folders to avoid re-scraping if the process stops.
+Implement configurable parallel processing (e.g., run 2, 5, 10+ simultaneous scrapes)
+Add strong error handling, retry logic, and request throttling.
+Ensure high speed with minimal failures.
**Technical Skills Needed
Python 3.10+
Requests or HTTPX
Captchha solving APIs
lxml or BeautifulSoup
Experience with sessions, cookies, and dynamic HTML parsing
Experience with multiprocessing or asyncio for large-scale scraping
Logging and fault-tolerant architecture
**Deliverables
Fully functional scraping bot with clean, documented Python code
Config file (parallelism, API keys, output folders)
Ability to continue from where it stopped (HTML caching)
Output in CSV/XLSX
Installation/execution guide
Optional: Standalone executable