Python Web Scraper Enhancement
Budget: $30 – $250 AUD
Project Overview
I have an existing Python web scraping setup that collects property listings from Portal Inmobiliario, a Chilean real estate site (in Spanish). The current code was developed by an English-speaking freelancer and works, but it's fragmented across multiple scripts and not fully production-ready.
________________________________________
Current Setup
• Input: A search results URL.
• Step 1: Identify all result pages (~50 properties per page), max of 42 pages that site allows.
• Step 2: Extract all property URLs from those pages.
• Step 3: Scrape each individual property page to extract attributes and save to an excel database.
• main.py only performs Step 3 and expects a list of property URLs as input.
• Steps 1 and 2 are implemented in two Jupyter Notebooks, I don’t know difference between main/ipynb and mainly.ipynb.
The scraper uses Selenium, since the target website is JavaScript-heavy.
Attached the work I was handed (Mr Eduardo.rar)
________________________________________
What I Need
1. Code Enhancement
1.a. Adapt the script to support apartment listings, not houses anymore.
Use this sample URL as the new search input:
https://www.portalinmobiliario.com/venta/departamento/metropolitana/_PriceRange_20000000CLP-400000000CLP_NoIndex_True#applied_filter_id%3Dprice%26applied_filter_name%3DPrecio%26applied_filter_order%3D6%26applied_value_id%3D20000000-400000000%26applied_value_name%3DCLP20000000-CLP400000000%26applied_value_order%3D4%26applied_value_results%3DUNKNOWN_RESULTS%26is_custom%3Dtrue
The site structure is mostly the same, but apartment listings may contain slightly different attributes than house listings, so the parser might need adjustments.
1.b. In addition to the existing scraped fields, also extract:
• Latitude and Longitude of each property.
• Date of Publication of the ad.
________________________________________
2. Productionization
Turn the script into a fully automated daily scraper, with the following features:
• Run once a day at a scheduled time.
• Scrape all apartment listings published today (using the above URL format).
• Append the extracted data to a persistent database.
• Log basic metrics (e.g. number of properties scraped, any errors).
________________________________________
Hosting & Deployment Requirements
• I’m agnostic on the hosting platform — open to Google Cloud, AWS, DigitalOcean, Railway, Render, etc.
1 That said, I’m leaning toward DigitalOcean Droplets due to:
1.a Low cost (as little as USD $4/month)
1.b Support for headless Chrome or Firefox (needed for Selenium)
1.c Sufficient specs (512MB RAM, 1 CPU, 10GB SSD is enough for now I believe)
2 Ideally, the whole setup (script + database + scheduler) should:
2.a Be cheap
2.b Require minimal maintenance
2.c Be scalable in case I want to scrape more property types later (e.g., rentals or commercial)
________________________________________
Deliverables
• Unified, clean and well commented Python script (no Jupyter notebooks)
• Works for apartments and captures the new fields
• Production-ready with: Daily scheduling, Database integration, Error handling/logging
• Deployment instructions (or optionally full setup done for me)
I have an existing Python web scraping setup that collects property listings from Portal Inmobiliario, a Chilean real estate site (in Spanish). The current code was developed by an English-speaking freelancer and works, but it's fragmented across multiple scripts and not fully production-ready.
________________________________________
Current Setup
• Input: A search results URL.
• Step 1: Identify all result pages (~50 properties per page), max of 42 pages that site allows.
• Step 2: Extract all property URLs from those pages.
• Step 3: Scrape each individual property page to extract attributes and save to an excel database.
• main.py only performs Step 3 and expects a list of property URLs as input.
• Steps 1 and 2 are implemented in two Jupyter Notebooks, I don’t know difference between main/ipynb and mainly.ipynb.
The scraper uses Selenium, since the target website is JavaScript-heavy.
Attached the work I was handed (Mr Eduardo.rar)
________________________________________
What I Need
1. Code Enhancement
1.a. Adapt the script to support apartment listings, not houses anymore.
Use this sample URL as the new search input:
https://www.portalinmobiliario.com/venta/departamento/metropolitana/_PriceRange_20000000CLP-400000000CLP_NoIndex_True#applied_filter_id%3Dprice%26applied_filter_name%3DPrecio%26applied_filter_order%3D6%26applied_value_id%3D20000000-400000000%26applied_value_name%3DCLP20000000-CLP400000000%26applied_value_order%3D4%26applied_value_results%3DUNKNOWN_RESULTS%26is_custom%3Dtrue
The site structure is mostly the same, but apartment listings may contain slightly different attributes than house listings, so the parser might need adjustments.
1.b. In addition to the existing scraped fields, also extract:
• Latitude and Longitude of each property.
• Date of Publication of the ad.
________________________________________
2. Productionization
Turn the script into a fully automated daily scraper, with the following features:
• Run once a day at a scheduled time.
• Scrape all apartment listings published today (using the above URL format).
• Append the extracted data to a persistent database.
• Log basic metrics (e.g. number of properties scraped, any errors).
________________________________________
Hosting & Deployment Requirements
• I’m agnostic on the hosting platform — open to Google Cloud, AWS, DigitalOcean, Railway, Render, etc.
1 That said, I’m leaning toward DigitalOcean Droplets due to:
1.a Low cost (as little as USD $4/month)
1.b Support for headless Chrome or Firefox (needed for Selenium)
1.c Sufficient specs (512MB RAM, 1 CPU, 10GB SSD is enough for now I believe)
2 Ideally, the whole setup (script + database + scheduler) should:
2.a Be cheap
2.b Require minimal maintenance
2.c Be scalable in case I want to scrape more property types later (e.g., rentals or commercial)
________________________________________
Deliverables
• Unified, clean and well commented Python script (no Jupyter notebooks)
• Works for apartments and captures the new fields
• Production-ready with: Daily scheduling, Database integration, Error handling/logging
• Deployment instructions (or optionally full setup done for me)
Related categories:
JavaScript
Python
Data Processing
Web Scraping
Django
Data Extraction
Data Analysis
Selenium
Automation
Database Management