Python Ads Scraping Automation

I’m building an internal library of marketing automations and need the first brick laid: a Python script that gathers competitor ad data straight from Google Search results and the Google Ads Transparency Center. The script has to run reliably on a Windows machine we keep online 24/7.

Here’s what I’m after:
• Pull ad headline, description, extensions, display & final URLs, advertiser/account info, first-seen/last-seen dates—and, if feasible, a screenshot of each ad.
• Handle pagination, “show more” buttons, and possible CAPTCHA interruptions with Playwright or Selenium; fall back on requests + BeautifulSoup whenever that’s faster.
• Rotate proxies and user-agents (I’ll provide lists) and expose all such settings in a simple config file.
• Drop the results into Postgres (CSV as interim output is fine) and ship a clean README plus a sample .bat/.ps1 so we can schedule the job in Windows Task Scheduler.
• Keep the code modular and well-commented; I’ll be extending it later to automate form submissions and trigger Google Ads API actions.

The finished project should arrive as a runnable repo (poetry or pip-tools), with unit tests for the parsing logic; Docker knowledge will come in handy for future phases but isn’t mandatory for this milestone.

When you respond, highlight experience on similar scraping or Google Ads projects and outline—in a few lines—how you’d tackle extracting data from the Transparency Center.