High-Performance Bulk Account Verifier & Classifier (bulks Records)
Budget: $250 – $750 USD
I need a robust, high-performance automation tool to verify and categorize approximately 300,000 user accounts from two databases. The tool will check login validity on a specific portal and classify accounts by type.
Key Technical Challenges:
Large Scale Processing: The tool must handle 300,000+ entries efficiently. I need a developer who understands multi-threading and asynchronous processing to ensure the task doesn't take weeks.
Smart De-duplication: There are many duplicates across the databases. The tool must include a pre-processing step to clean and de-duplicate the list before starting the verification process to save time and resources.
Account Classification: For every successful login, the tool must scrape the account dashboard to identify if it is a Corporate (Enterprise) or Individual account.
Requirements:
Efficiency & Speed: Use of Python (Asyncio/Aiohttp), Node.js, or Go is preferred for high-concurrency.
Proxy Support: Since we are checking 300k accounts, the tool must support HTTP/SOCKS5 proxies (with rotation) to avoid IP bans.
Database Management: Ability to handle large SQL/NoSQL exports and merge results without data loss.
Error Handling: Robust "retry" logic for timed-out connections or portal glitches.
Deliverables:
A high-speed verification engine.
A clean, de-duplicated final database/report with Status (Live/Dead) and Type (Corporate/Individual).
Source code with instructions on how to manage proxy integration.
Key Technical Challenges:
Large Scale Processing: The tool must handle 300,000+ entries efficiently. I need a developer who understands multi-threading and asynchronous processing to ensure the task doesn't take weeks.
Smart De-duplication: There are many duplicates across the databases. The tool must include a pre-processing step to clean and de-duplicate the list before starting the verification process to save time and resources.
Account Classification: For every successful login, the tool must scrape the account dashboard to identify if it is a Corporate (Enterprise) or Individual account.
Requirements:
Efficiency & Speed: Use of Python (Asyncio/Aiohttp), Node.js, or Go is preferred for high-concurrency.
Proxy Support: Since we are checking 300k accounts, the tool must support HTTP/SOCKS5 proxies (with rotation) to avoid IP bans.
Database Management: Ability to handle large SQL/NoSQL exports and merge results without data loss.
Error Handling: Robust "retry" logic for timed-out connections or portal glitches.
Deliverables:
A high-speed verification engine.
A clean, de-duplicated final database/report with Status (Live/Dead) and Type (Corporate/Individual).
Source code with instructions on how to manage proxy integration.
Related categories:
Python
Data Processing
SQL
Web Scraping
Software Architecture
Node.js
NoSQL
Automation
Database Management
API Development