Live Data Scraping & Code Optimisation

I have an existing Next.js / React front-end that already contains scraping logic, but it needs a seasoned hand to refine it, extend it and keep it running smoothly. The core task is to pull live textual data on demand from several sources—traditional websites, public or private APIs, and a couple of internal databases—and present it instantly to users.

I’ll rely on you to audit the current codebase, identify bottlenecks, tighten performance, and add fresh scrapers where gaps exist. Robust error handling, rate-limit management and stealth techniques are essential because I can’t afford unexpected downtime. When things break (and they occasionally will), I need someone who can step in quickly, debug, and ship a fix without fuss.

This is an ongoing collaboration, so clean, well-documented code and clear commit messages are non-negotiable. Please tell me about projects where you solved similar scraping or optimisation challenges with JavaScript, TypeScript, Next.js, React, Node, Puppeteer, Playwright, Cheerio, or comparable tools. If you thrive on squeezing every millisecond out of a build and enjoy turning messy data sources into reliable endpoints, let’s talk.