Expert Python Data Scraper

Job ID: 40215999

Budget: $30 – $250 USD

I need a seasoned Python developer to build a robust scraper that collects the required data and writes it straight to JSON—no additional cleaning or processing necessary. Once we begin I’ll provide the target URL(s) and any access details; for now, assume a standard public site with pagination and occasional anti-bot checks.

Core expectations
• Written in Python 3 using requests/BeautifulSoup or Scrapy; resort to Selenium only if there’s no lighter workaround.
• Handles pagination, retries, and polite delays gracefully so the run can complete unattended.
• Config file or clear constants for headers, cookies, and start URLs, letting me tweak targets without editing core logic.
• Produces a single JSON file (or one file per page if that’s cleaner) reflecting the raw content exactly as captured.

Deliverables
1. Fully commented source code.
2. README with quick-start setup and sample command.
3. Example JSON output generated by the script.

If you’ve tackled heavy-duty scraping before and can turn this around quickly, I’d love to hear your approach and estimated timeline.