Scrape 84k Protected Records
Budget: $250 – $750 AUD
I have a website that sits behind a standard username-password login and hosts exactly 84,452 text-based records. My goal is to capture every record in a clean, structured format so I can run analysis offline.
After I share credentials and show you the page flow, I’ll confirm which fields matter most, but they are plain text only—no images or binary files. Pagination and a couple of dynamic filters appear on the site, so the scraper needs to navigate those automatically, respect any anti-bot measures, and avoid missing or duplicating entries.
The end product I expect is a tidy CSV (or Excel if you prefer) containing one row per record with all agreed-upon columns populated and verified. If you can include basic error logging and a rerun mechanism in case the session times out, that’s a plus.
After I share credentials and show you the page flow, I’ll confirm which fields matter most, but they are plain text only—no images or binary files. Pagination and a couple of dynamic filters appear on the site, so the scraper needs to navigate those automatically, respect any anti-bot measures, and avoid missing or duplicating entries.
The end product I expect is a tidy CSV (or Excel if you prefer) containing one row per record with all agreed-upon columns populated and verified. If you can include basic error logging and a rerun mechanism in case the session times out, that’s a plus.