Web Scraping | Data Mining
Budget: $250 – $750 USD
Job Description
- We are looking a developer who can build a custom software to automate scraping of a website on specific dates/days
- We want to scrape very specific infomation (per second and minute respectively) from the site when the site is live. (Information on the website will be updated by the seconds when the site is live.)
Couple of challenges with this site.
1. It has a login with verification code.
2. The login is a popup new window.
3. After login there is a 4 (personal) pin code to enter, which is a new window.
4. The final logged in state will be in a popup new window.
5. The website will ban the account if it finds multiple accounts are accessed from the same IP.
Couple of requirements
1. We would require the developer to be able to double check the software is working as intended before the specific live dates/days. A separate monthly fee for weekly maintenance to be discussed.
2. To patch or fix the software when the site changes or updates thier website code.
3. A notification alert if any of the specific information fails to be scraped.
4. Prefer to be able to view the consolidated info in our own PC.
We are ok to
1. Manually login and then only allow the software to scrape.
2. No preference on the programming language.
3. We just need to see accurate & consolidated information of the specific information.
More details upon discussion. Cheers
- We are looking a developer who can build a custom software to automate scraping of a website on specific dates/days
- We want to scrape very specific infomation (per second and minute respectively) from the site when the site is live. (Information on the website will be updated by the seconds when the site is live.)
Couple of challenges with this site.
1. It has a login with verification code.
2. The login is a popup new window.
3. After login there is a 4 (personal) pin code to enter, which is a new window.
4. The final logged in state will be in a popup new window.
5. The website will ban the account if it finds multiple accounts are accessed from the same IP.
Couple of requirements
1. We would require the developer to be able to double check the software is working as intended before the specific live dates/days. A separate monthly fee for weekly maintenance to be discussed.
2. To patch or fix the software when the site changes or updates thier website code.
3. A notification alert if any of the specific information fails to be scraped.
4. Prefer to be able to view the consolidated info in our own PC.
We are ok to
1. Manually login and then only allow the software to scrape.
2. No preference on the programming language.
3. We just need to see accurate & consolidated information of the specific information.
More details upon discussion. Cheers