Scraping Real estate website (100$) -- 2
Budget: $30 – $250 AUD
I need to extract information from a real-estate website and pack them into an excel sheet or a database.
The site name is domain.com.au
The extracted information should include the following :
(1) suburb profiles: https://www.domain.com.au/suburb-profile/point-cook-vic-3030
(2) all the properties listed on the site for sale and the details of each property
https://www.domain.com.au/sale/point-cook-vic-3030/?excludeunderoffer=1
(3) all the properties listed on the site for rent and the details of each property
https://www.domain.com.au/rent/point-cook-vic-3030/?excludedeposittaken=1
(4) The history for each property from the URL below:
https://www.domain.com.au/property-profile/18-treeleaf-avenue-point-cook-vic-3030
The deliverable will include the following:
1. The suburb data in an excel sheet
2. The sale property data in an excel sheet (including the property history from point#4)
3. The rent property data in an excel sheet (including the property history from point#4)
4. The code for scraping the data and storing them locally in python
5. An API that takes the URL as an input and returns the values of all fields in that page (the page will be a property listing for sale or rent similar to URLS in points 2 and 3)
I attached samples of the suburb_profile, sales_infor and suburbs listing
The code should be smart enough to avoid being blocked by the site servers, this might require adding lags between every request and the other, and may require hiding the identity of the request issuer by hiding the IP or using a proxy chain
The site name is domain.com.au
The extracted information should include the following :
(1) suburb profiles: https://www.domain.com.au/suburb-profile/point-cook-vic-3030
(2) all the properties listed on the site for sale and the details of each property
https://www.domain.com.au/sale/point-cook-vic-3030/?excludeunderoffer=1
(3) all the properties listed on the site for rent and the details of each property
https://www.domain.com.au/rent/point-cook-vic-3030/?excludedeposittaken=1
(4) The history for each property from the URL below:
https://www.domain.com.au/property-profile/18-treeleaf-avenue-point-cook-vic-3030
The deliverable will include the following:
1. The suburb data in an excel sheet
2. The sale property data in an excel sheet (including the property history from point#4)
3. The rent property data in an excel sheet (including the property history from point#4)
4. The code for scraping the data and storing them locally in python
5. An API that takes the URL as an input and returns the values of all fields in that page (the page will be a property listing for sale or rent similar to URLS in points 2 and 3)
I attached samples of the suburb_profile, sales_infor and suburbs listing
The code should be smart enough to avoid being blocked by the site servers, this might require adding lags between every request and the other, and may require hiding the identity of the request issuer by hiding the IP or using a proxy chain