Real-time data bot to scrape live data
Budget: $30 – $250 USD
So this is what i need. I don’t know what it is called . Maybe a scraper or a bot or just automation.
We have the php script that currently takes raw data and outputs it to a url . We basically can use that however I have a new thing to add. The old script went to single url.
To make it faster and more accurate and efficient what I want to do is to monitor several websites like: google, yahoo , fox, and so on. It would monitor all of them and as soon as it noticed a change on any of these sites it would then post it to our output page just like before
Now I don’t know how this works or if you can do it. My questions does the bot keep calling each webpage over and over? Or does it just listen and when there is a change then it calls it?
What I don’t want is my ip getting blocked from any of these sites for making too many calls in one day.
However to be most accurate we need to check every second so it is nearly instant when there is a change..
Need to do this right away. Must be willing to work on long term project
Important to read my budget is $100-$150. If we can show then not is working we could increase the budget to meet more design needs if necessary
We have the php script that currently takes raw data and outputs it to a url . We basically can use that however I have a new thing to add. The old script went to single url.
To make it faster and more accurate and efficient what I want to do is to monitor several websites like: google, yahoo , fox, and so on. It would monitor all of them and as soon as it noticed a change on any of these sites it would then post it to our output page just like before
Now I don’t know how this works or if you can do it. My questions does the bot keep calling each webpage over and over? Or does it just listen and when there is a change then it calls it?
What I don’t want is my ip getting blocked from any of these sites for making too many calls in one day.
However to be most accurate we need to check every second so it is nearly instant when there is a change..
Need to do this right away. Must be willing to work on long term project
Important to read my budget is $100-$150. If we can show then not is working we could increase the budget to meet more design needs if necessary