Automated Daily Email Scraper
Budget: ₹1,500 – ₹12,500 INR
I need a reliable developer to build a fully-automated system that gathers fresh, publicly listed email addresses from Google search results (pulled through a SERP API or an equivalent method) every single day, verifies them for validity, and delivers a clean CSV ready for my marketing campaigns.
Here’s what I’m after:
• Workflow
1. Submit a set of keywords or niche phrases.
2. Crawl the top Google result pages returned by a SERP service, extract any visible email addresses, and capture the source URL and page title.
3. Run each address through an SMTP-level verifier (ZeroBounce, NeverBounce, or an in-house Python verifier—whichever you prefer, as long as it returns status codes for valid, invalid, catch-all, disposable, and role accounts).
4. Output only “valid” or “catch-all” emails in a downloadable CSV along with their metadata.
• Technical notes
- I’m comfortable with Python (Scrapy, Requests, BeautifulSoup, Selenium) or Node (Puppeteer, Cheerio); choose whichever stack you can scale and maintain.
- Respect Google’s ToS with rotating residential proxies or a paid SERP API to avoid blocking and captchas.
- The job should run via a daily cron or cloud function and log results to a lightweight dashboard (even a simple Flask/Express UI or Google Sheet is fine).
- Code must be modular so I can swap in new keywords, adjust verification thresholds, or change the SERP provider without rewriting everything.
Acceptance criteria
• Daily schedule triggers without manual intervention.
• Minimum 95 % deliverability rate on the “valid” list according to the verifier report.
• Duplicate and role-based addresses removed automatically.
• Well-documented README plus a short Loom video walking through setup and adding new keywords.
If you’ve built similar data pipelines or have clever ideas on keeping the scrape respectful yet thorough, I’m all ears.
Here’s what I’m after:
• Workflow
1. Submit a set of keywords or niche phrases.
2. Crawl the top Google result pages returned by a SERP service, extract any visible email addresses, and capture the source URL and page title.
3. Run each address through an SMTP-level verifier (ZeroBounce, NeverBounce, or an in-house Python verifier—whichever you prefer, as long as it returns status codes for valid, invalid, catch-all, disposable, and role accounts).
4. Output only “valid” or “catch-all” emails in a downloadable CSV along with their metadata.
• Technical notes
- I’m comfortable with Python (Scrapy, Requests, BeautifulSoup, Selenium) or Node (Puppeteer, Cheerio); choose whichever stack you can scale and maintain.
- Respect Google’s ToS with rotating residential proxies or a paid SERP API to avoid blocking and captchas.
- The job should run via a daily cron or cloud function and log results to a lightweight dashboard (even a simple Flask/Express UI or Google Sheet is fine).
- Code must be modular so I can swap in new keywords, adjust verification thresholds, or change the SERP provider without rewriting everything.
Acceptance criteria
• Daily schedule triggers without manual intervention.
• Minimum 95 % deliverability rate on the “valid” list according to the verifier report.
• Duplicate and role-based addresses removed automatically.
• Well-documented README plus a short Loom video walking through setup and adding new keywords.
If you’ve built similar data pipelines or have clever ideas on keeping the scrape respectful yet thorough, I’m all ears.
Related categories:
PHP
Python
Web Scraping
Software Architecture
Node.js
Web Development
Scrapy
Data Extraction
Automation
API Integration