Real Estate RETS Data & PostgreSQL API
Budget: €12 – €18 EUR
1. Review an existing PostgreSQL database that was not completely setup this project.
2. Use RETS system to download 20 years worth of historical data from an existing real estate MLS server.
3. Use RESTFUL API to set up daily, incremental downloads of the same data.
The existing database is a real estate MLS using a very well established RESTFUL API standard called RESO Web API Core 1.0 (https://www.reso.org/reso-web-api/).
The reason for using the RETS system to do the initial download is that the number of historical records in some of the data fields has proven to be impractical (tens of millions of records) to download using the API as it is simply too slow. I am open to alternative methods here though.
There will be a limited number of transforms (calculated fields) based upon the existing data structure, perhaps 15-20.
Ideally, once the database is set up, I'd like to create a system (Excel or custom) to connect to the database and run a series of statistical queries on it. That does not need to be part of the original scope of work, but if the same developer can assist with this part that would be great.
A previous freelancer started this project. They set up the database and tried using the API to do the initial download. Most of the data tables were able to be retrieved fully this way, however the largest tables were not. That freelancer provided documentation on the system they built, that will be provided in full to the new freelancer chosen to finish the project.
The existing system is build primarily with Python. And an existing GitHub repository is available at https://github.com/slobodadev/hagan_realty
The end goal of this system is to have all of the data available locally for us to run statistical analysis on it. That will be a second phase of this project and is not included in this specific job.
2. Use RETS system to download 20 years worth of historical data from an existing real estate MLS server.
3. Use RESTFUL API to set up daily, incremental downloads of the same data.
The existing database is a real estate MLS using a very well established RESTFUL API standard called RESO Web API Core 1.0 (https://www.reso.org/reso-web-api/).
The reason for using the RETS system to do the initial download is that the number of historical records in some of the data fields has proven to be impractical (tens of millions of records) to download using the API as it is simply too slow. I am open to alternative methods here though.
There will be a limited number of transforms (calculated fields) based upon the existing data structure, perhaps 15-20.
Ideally, once the database is set up, I'd like to create a system (Excel or custom) to connect to the database and run a series of statistical queries on it. That does not need to be part of the original scope of work, but if the same developer can assist with this part that would be great.
A previous freelancer started this project. They set up the database and tried using the API to do the initial download. Most of the data tables were able to be retrieved fully this way, however the largest tables were not. That freelancer provided documentation on the system they built, that will be provided in full to the new freelancer chosen to finish the project.
The existing system is build primarily with Python. And an existing GitHub repository is available at https://github.com/slobodadev/hagan_realty
The end goal of this system is to have all of the data available locally for us to run statistical analysis on it. That will be a second phase of this project and is not included in this specific job.