Build scrapy-redis web scraping environment
Budget: $250 – $750 USD
This is a test project to determine feasibility of scrapy-redis.
Project: Leverage scrapy-redis to scrape keywords from an e-commerce website.
1) read DB table to create inputs for scrapy-redis spider
2) distribute inputs through scrap-redis infrastructure
3) output scrapy results to DB
You will be provided a current working version of a scrapy spider in python3.x.
You will need to install / configure scrapy-redis on test ubuntu server.
You will access existing DB table to construct inputs for scrapy-redis.
You will output spider results to new database table.
You will fully document the steps needed to modify/congfigure existing spider to work with scrapy-redis.
Please do not apply if you are not familiar with scrapy-redis and its implementation. I am looking for someone I can converse with and learn the installation and configuration.
Project: Leverage scrapy-redis to scrape keywords from an e-commerce website.
1) read DB table to create inputs for scrapy-redis spider
2) distribute inputs through scrap-redis infrastructure
3) output scrapy results to DB
You will be provided a current working version of a scrapy spider in python3.x.
You will need to install / configure scrapy-redis on test ubuntu server.
You will access existing DB table to construct inputs for scrapy-redis.
You will output spider results to new database table.
You will fully document the steps needed to modify/congfigure existing spider to work with scrapy-redis.
Please do not apply if you are not familiar with scrapy-redis and its implementation. I am looking for someone I can converse with and learn the installation and configuration.