Pull data from API and import to MySql every hour with some logic requirements
Budget: $250 – $750 CAD
I need to pull data from an API every hour and have it go to a MySQL database. The API connection works fine and is an established system. I have not set anything up yet in MySQL. Here's what I require:
The database needs to be like a Master List of all properties. When it pulls the data every hour it should add any rows that do not yet exist based on the unique key field called ListingID. If the row already exists, there is a field that indicates when the data in that row was last modified so if the data has been updated and the two dates don't match then the row should be updated. If the last update matches the date in the database then no need to save the data for that row. That's pretty much it for the Master List.
I will need secondary tables that pull from the primary list based on a different key field called DestinationId. Kind of like providing filtered rows to multiple other tables. When it copies from the master list to the other tables, the subtables need some extra columns to track when the price has been modified from an old price to a new price and same thing for certain other fields, only a few.
The exact nature of my project is that I'm a realtor and I have access to a data feed of all the real estate listings as a Master List. I work with other agents who need their own subset of specific rows from the master list imported to their own database table. Some properties will be copied to different sub table based on the DestinationId. From there, my integration can detect the change through the restAPI for MySQL and will trigger several automations. I can probably explain and demonstrate what I need better over a Zoom meeting.
The database needs to be like a Master List of all properties. When it pulls the data every hour it should add any rows that do not yet exist based on the unique key field called ListingID. If the row already exists, there is a field that indicates when the data in that row was last modified so if the data has been updated and the two dates don't match then the row should be updated. If the last update matches the date in the database then no need to save the data for that row. That's pretty much it for the Master List.
I will need secondary tables that pull from the primary list based on a different key field called DestinationId. Kind of like providing filtered rows to multiple other tables. When it copies from the master list to the other tables, the subtables need some extra columns to track when the price has been modified from an old price to a new price and same thing for certain other fields, only a few.
The exact nature of my project is that I'm a realtor and I have access to a data feed of all the real estate listings as a Master List. I work with other agents who need their own subset of specific rows from the master list imported to their own database table. Some properties will be copied to different sub table based on the DestinationId. From there, my integration can detect the change through the restAPI for MySQL and will trigger several automations. I can probably explain and demonstrate what I need better over a Zoom meeting.