Cronjob for website data extract
Budget: $30 – $250 USD
I am looking for a freelancer who can create a cronjob for extracting data from a website. The ideal candidate should have experience in web scraping and cronjob automation.
Source of Data:
- The data to be extracted is from a website (realtaxdeed.com)
- there are 82 pages in the website, for each county real estate auction (in the website, check the "Jump to" item in the menu)
- in each page, there is a calendar of scheduled auctions for forecluse and tax deed auctions
The list of properties for each auction (with their corresponding data) should be extracted in a daily basis and imported into the MySQL database. The data to be extracted is related to auctions for next 3 months.
The date to be extracted is:
- auction date & time
- auction type
- case # (and its hyperlink, if any)
- certificate # (and its hyperlink, if any)
- opening bid
- Parcel Id (and its hyperlink, if any)
- Property Address
- Assessed Value
The same extract process can be used for all counties and auctions, as the pages are the same. The cron job will need to run for all auctions on the 82 pages.
It should be configurable to include/remove pages, if new counties are added on the realtaxdeed.com platform.
Data Format:
- The extracted data should be in JSON format.
- The data should be stored in a MySQL database
Cronjob Frequency:
- The cronjob should run every day.
Skills and Experience:
- Proficiency in web scraping techniques and tools.
- Experience in setting up and automating cronjobs.
- Familiarity with JSON data format.
- Experience with MySQL database.
Please include your relevant experience and examples of previous similar projects in your proposal.
Source of Data:
- The data to be extracted is from a website (realtaxdeed.com)
- there are 82 pages in the website, for each county real estate auction (in the website, check the "Jump to" item in the menu)
- in each page, there is a calendar of scheduled auctions for forecluse and tax deed auctions
The list of properties for each auction (with their corresponding data) should be extracted in a daily basis and imported into the MySQL database. The data to be extracted is related to auctions for next 3 months.
The date to be extracted is:
- auction date & time
- auction type
- case # (and its hyperlink, if any)
- certificate # (and its hyperlink, if any)
- opening bid
- Parcel Id (and its hyperlink, if any)
- Property Address
- Assessed Value
The same extract process can be used for all counties and auctions, as the pages are the same. The cron job will need to run for all auctions on the 82 pages.
It should be configurable to include/remove pages, if new counties are added on the realtaxdeed.com platform.
Data Format:
- The extracted data should be in JSON format.
- The data should be stored in a MySQL database
Cronjob Frequency:
- The cronjob should run every day.
Skills and Experience:
- Proficiency in web scraping techniques and tools.
- Experience in setting up and automating cronjobs.
- Familiarity with JSON data format.
- Experience with MySQL database.
Please include your relevant experience and examples of previous similar projects in your proposal.