Custom Keyword-Scraper Dev (Python/C#)
Budget: $8 – $15 USD
Keyword Scraper (Python or C#) Google"
I am looking for an experienced developer to create a custom Keyword Scraper that can search and collect links based on specific profiles. This project can be developed in Python or C#, depending on your preference and expertise.
Project Requirements:
Keyword-Based Scraping:
Main Keywords: URLs only, exact match.
Secondary Keywords: Titles, descriptions, URLs, metadata.
Custom Keywords: Full-text search with domain targeting.
Domain Filtering:
Use domain lists like cache.txt, source.txt, leaks.txt to filter and organize URLs.
whitelist.txt to exclude specific domains from scans.
Profile-Based Output:
Each profile should have separate folders for main, secondary, and custom keywords.
Each folder should contain cache.txt, source.txt, leaks.txt, and new.txt outputs.
Automatic duplicate removal across all output files.
Proxy and Fingerprint Rotation:
Use random fingerprints for each session.
Support for HTTP and SOCKS5 proxies with ip:port:user:pass authentication.
Operating Modes:
Real-Time Mode: Use Selenium for live, visible link collection.
Headless Mode: For faster, silent scraping without visible browser interaction.
Efficiency and Performance:
Multi-threaded or asynchronous processing for faster scans.
Error handling and retry logic for failed connections.
Additional Features:
Configurable delay settings to avoid rate-limiting.
Support for multi-profile scanning at the same time.
Clean and well-documented code for easy maintenance and future updates.
I am looking for an experienced developer to create a custom Keyword Scraper that can search and collect links based on specific profiles. This project can be developed in Python or C#, depending on your preference and expertise.
Project Requirements:
Keyword-Based Scraping:
Main Keywords: URLs only, exact match.
Secondary Keywords: Titles, descriptions, URLs, metadata.
Custom Keywords: Full-text search with domain targeting.
Domain Filtering:
Use domain lists like cache.txt, source.txt, leaks.txt to filter and organize URLs.
whitelist.txt to exclude specific domains from scans.
Profile-Based Output:
Each profile should have separate folders for main, secondary, and custom keywords.
Each folder should contain cache.txt, source.txt, leaks.txt, and new.txt outputs.
Automatic duplicate removal across all output files.
Proxy and Fingerprint Rotation:
Use random fingerprints for each session.
Support for HTTP and SOCKS5 proxies with ip:port:user:pass authentication.
Operating Modes:
Real-Time Mode: Use Selenium for live, visible link collection.
Headless Mode: For faster, silent scraping without visible browser interaction.
Efficiency and Performance:
Multi-threaded or asynchronous processing for faster scans.
Error handling and retry logic for failed connections.
Additional Features:
Configurable delay settings to avoid rate-limiting.
Support for multi-profile scanning at the same time.
Clean and well-documented code for easy maintenance and future updates.