Scrapy Crawler/Scraper (Scrapy Experience Required, Ongoing Work)
Budget: $25 – $50 USD
I am looking for an experienced python developer who has hands-on knowledge of Scrapy and the Scrapy ecosystem (Scrapyd, Splash, Crawlera, etc...) to build out a large scraping platform (multiple data sources, some used to establish reference data, others used to fetch timely information).
Details of this first phase of the project is contained in the following document:
https://docs.google.com/document/d/1Wjdx0zeY4vzlaHR5p81I8KhhfObwFgghbkd2Rn4cf-4/edit?usp=sharing
Fetching pages like these:
https://www.marketwatch.com/investing/stock/efsh
https://www.marketwatch.com/investing/stock/egp
https://www.marketwatch.com/investing/stock/aapl
https://www.marketwatch.com/investing/stock/msft
And extracting basic data, mostly from the page's meta tags.
Looking to get started asap. Please do not respond if you do not have experience with Scrapy.
THIS IS GOING TO BE ONGOING WORK - WHICH IS WHY I POSTED AN HOURLY PROJECT. LOOKING TO SPEND MAX $35/HR. PLEASE INCLUDE IN YOUR RESPONSE YOUR HOUR ESTIMATE FOR THIS FIRST PHASE OF THE PROJECT.
Details of this first phase of the project is contained in the following document:
https://docs.google.com/document/d/1Wjdx0zeY4vzlaHR5p81I8KhhfObwFgghbkd2Rn4cf-4/edit?usp=sharing
Fetching pages like these:
https://www.marketwatch.com/investing/stock/efsh
https://www.marketwatch.com/investing/stock/egp
https://www.marketwatch.com/investing/stock/aapl
https://www.marketwatch.com/investing/stock/msft
And extracting basic data, mostly from the page's meta tags.
Looking to get started asap. Please do not respond if you do not have experience with Scrapy.
THIS IS GOING TO BE ONGOING WORK - WHICH IS WHY I POSTED AN HOURLY PROJECT. LOOKING TO SPEND MAX $35/HR. PLEASE INCLUDE IN YOUR RESPONSE YOUR HOUR ESTIMATE FOR THIS FIRST PHASE OF THE PROJECT.