Scrape all website's links
Budget: $30 – $250 USD
I would like to hire someone to build a program that will find all links under a website that doesn't have a sitemap or anything like that.
For example if I give you the root website http://www.example.com
I would like the program to return http://www.example.com/file/200, http://www.example.com/file/about-us etc. even if the homepage does not link to any other page.
If it can be done in python, that would be ideal.
Thank you.
For example if I give you the root website http://www.example.com
I would like the program to return http://www.example.com/file/200, http://www.example.com/file/about-us etc. even if the homepage does not link to any other page.
If it can be done in python, that would be ideal.
Thank you.