Python Data Scrape with Scrapy
Budget: $10 – $30 USD
I am looking for a skillful Python developer with proficiency in using Scrapy for web scraping. The task will involve generating a python script to scrape websites for data.
Key requirements:
- Use Scrapy.org as the main tool for this work.
- Ability to scrape/crawl full websites.
- Make a sitemap excel file from the crawled website with all links parent, children, grandchildren
- download the full website to a folder
- convert the downloaded pages to text file using html2text in a new folder
Experience in web scraping and data extraction is essential. Familiarity with handling diverse websites' structures will be a plus. Regardless of the data structure, your code should be clean, easy to understand, and efficient. Let's pull the web apart!
Key requirements:
- Use Scrapy.org as the main tool for this work.
- Ability to scrape/crawl full websites.
- Make a sitemap excel file from the crawled website with all links parent, children, grandchildren
- download the full website to a folder
- convert the downloaded pages to text file using html2text in a new folder
Experience in web scraping and data extraction is essential. Familiarity with handling diverse websites' structures will be a plus. Regardless of the data structure, your code should be clean, easy to understand, and efficient. Let's pull the web apart!