Web scraping - Python / Scrapy
Budget: ₹1,500 – ₹12,500 INR
I have three websites for which I want to supply search input parameters and extract pricing data. The high level requirements steps are:
Python function accepts search input parameters
Search performed on three separate websites
Predefined information extracted from each website
Results collated and returned to the Python function
Bidders who demonstrate the required technical knowledge and a price within budget will receive a full description of the requirement. DO NOT SEND A TEMPLATE or A.I. RESPONSE, it will be ignored.
Technical Requirements
Python based
Scraper must simulate normal user usage. No getting banned!
A degree of adaptability to changes in output structure
Results will be spread over multiple pages. The scraper will need to be smart enough to simulate clicking the next page
Deliverables
Python, Scrapy based project which provides the main Python function
Test code
Support to setup my local development environment
Timeline
I would estimate the project will take no more than 2 days to complete. My estimate is based on 20+ years as a developer myself. I would consider well structured, qualified challenges from demonstrably experienced resources
My questions for you
Sorry but too many people just issue a template response of a AI generated text. I understand bidding for jobs can be a numbers game but we receive literally dozens of replies to every job and most are not suitable. With that in mind here are a few questions. Just a brief couple of sentences is fine.
1 - Describe your experience do you have with website scraping?
2 - Describe your experience with Scrapy specifically?
3 - Please confirm you understand the requirements, timeline and are submitting a bid for a fixed price? I will send more details to qualified bidders. Neither of us are committing to the job at this stage.
Python function accepts search input parameters
Search performed on three separate websites
Predefined information extracted from each website
Results collated and returned to the Python function
Bidders who demonstrate the required technical knowledge and a price within budget will receive a full description of the requirement. DO NOT SEND A TEMPLATE or A.I. RESPONSE, it will be ignored.
Technical Requirements
Python based
Scraper must simulate normal user usage. No getting banned!
A degree of adaptability to changes in output structure
Results will be spread over multiple pages. The scraper will need to be smart enough to simulate clicking the next page
Deliverables
Python, Scrapy based project which provides the main Python function
Test code
Support to setup my local development environment
Timeline
I would estimate the project will take no more than 2 days to complete. My estimate is based on 20+ years as a developer myself. I would consider well structured, qualified challenges from demonstrably experienced resources
My questions for you
Sorry but too many people just issue a template response of a AI generated text. I understand bidding for jobs can be a numbers game but we receive literally dozens of replies to every job and most are not suitable. With that in mind here are a few questions. Just a brief couple of sentences is fine.
1 - Describe your experience do you have with website scraping?
2 - Describe your experience with Scrapy specifically?
3 - Please confirm you understand the requirements, timeline and are submitting a bid for a fixed price? I will send more details to qualified bidders. Neither of us are committing to the job at this stage.
Related categories:
Business, Accounting, Human Resources & Legal
Python
Web Scraping
Software Architecture
Data Mining