Python Developer for Scrapy Property Data Extraction
Budget: $250 – $750 USD
We are looking for an experienced Python developer to create a web scraping script using Scrapy. The script should scrape property data from Idealista and extract detailed information for all types of properties.
Framework: The project must use the Python framework Scrapy.
Key Data to Scrape:
- Title of the listing
- Price of the property
- Location (address, neighborhood, city)
- Property features (size, bedrooms, bathrooms, type, orientation, floor)
- Description of the property
- Highlighted features (year of construction, energy rating, heating, elevator)
- Image URLs
- Floor plan (image or URL if available)
- Extras (storage, garage, pool, common areas)
- Contact information (name or agency and contact details)
- Environment (proximity to transport, schools, parks, map location)
- Listing status (reserved, sold, etc.)
- Additional costs (community fees, taxes)
Script Features:
- Handle pagination to scrape all properties from a URL
- Accept multiple base URLs like:
https://www.idealista.com/venta-viviendas/madrid-provincia/
https://www.idealista.com/venta-viviendas/barcelona-barcelona/
- Output data as Scrapy Items in JSON, CSV, or database format
Testing Example:
The script will be tested with https://www.idealista.com/venta-viviendas/barcelona-barcelona/
The script must scrape approximately 11,000 listings to be accepted.
Proxies:
If proxies are needed, integrate proxy usage (e.g., rotating proxies).
Inform us of any additional proxy costs.
Deliverables:
Fully functioning Scrapy script in Python
Documentation to run the script and configure URLs and proxies
Assistance during testing to verify the script works correctly
Important Note:
Payment will only be made if the script scrapes all required information and handles large-scale scraping.
Failure to scrape the ~11,000 listings from the test URL will result in no payment.
Framework: The project must use the Python framework Scrapy.
Key Data to Scrape:
- Title of the listing
- Price of the property
- Location (address, neighborhood, city)
- Property features (size, bedrooms, bathrooms, type, orientation, floor)
- Description of the property
- Highlighted features (year of construction, energy rating, heating, elevator)
- Image URLs
- Floor plan (image or URL if available)
- Extras (storage, garage, pool, common areas)
- Contact information (name or agency and contact details)
- Environment (proximity to transport, schools, parks, map location)
- Listing status (reserved, sold, etc.)
- Additional costs (community fees, taxes)
Script Features:
- Handle pagination to scrape all properties from a URL
- Accept multiple base URLs like:
https://www.idealista.com/venta-viviendas/madrid-provincia/
https://www.idealista.com/venta-viviendas/barcelona-barcelona/
- Output data as Scrapy Items in JSON, CSV, or database format
Testing Example:
The script will be tested with https://www.idealista.com/venta-viviendas/barcelona-barcelona/
The script must scrape approximately 11,000 listings to be accepted.
Proxies:
If proxies are needed, integrate proxy usage (e.g., rotating proxies).
Inform us of any additional proxy costs.
Deliverables:
Fully functioning Scrapy script in Python
Documentation to run the script and configure URLs and proxies
Assistance during testing to verify the script works correctly
Important Note:
Payment will only be made if the script scrapes all required information and handles large-scale scraping.
Failure to scrape the ~11,000 listings from the test URL will result in no payment.