Browser based WebCrawler
Budget: $30 – $250 USD
Hi,
I need a golang web crawler. It should take the website as input & crawl all the pages of the website & gather the following data in a MySQL database.
1. User ID, Website, URL, Page Title, Linked pages ( URLS), Linked Page (title).
The script should be hosted on AWS & should be scalable.
You would need to content the script/program with a website to take the input through the API.
Please let me know if you have expertise in building Crawler & how would you cope with our IPs getting blocked.
Please only mention the actual bid & time you would need. I'd be happy to discuss this with any genuine developer.
I need a golang web crawler. It should take the website as input & crawl all the pages of the website & gather the following data in a MySQL database.
1. User ID, Website, URL, Page Title, Linked pages ( URLS), Linked Page (title).
The script should be hosted on AWS & should be scalable.
You would need to content the script/program with a website to take the input through the API.
Please let me know if you have expertise in building Crawler & how would you cope with our IPs getting blocked.
Please only mention the actual bid & time you would need. I'd be happy to discuss this with any genuine developer.