Google maps scraper
Budget: $250 – $750 CAD
We have a google map scraper extension, which needs to be modified a bit in order to increase its accuracy and functionality.
The scraper has field to enter multiple keywords and it scrapes the page one by one for the mentioned keywords in order. Once a keyword is completed, the entry file automatically gets downloaded and it moves to next search.
But currently while scraping, initially the speed of scraping is good and it scrapes like 20 entries in 2-3 seconds. But after sometime like 30-45 minutes of scraping the speed gradually reduces to like taking 2-3 seconds for 1 entry.
What we think is that maybe due to high amount of entry accumulation in the page, it gets slowed down. So maybe if we clear the entries automatically (which currently isnt done) while moving from one keyword to another, it wont get slowed and the file will be downloaded as it is done for every keyword.
Also, sometimes while scraping, the page gets stuck on loading and the scraper thinks that entries are finished and hence it moves to next keyword.
But I observed that in order to load it further, we need to click on any entry and then click on back button, after which, the page gets loaded further. So we need to add a command, which would make the scraper click on any of the entries and then on back button before moving to the next keyword. So this way if there are more entries, then they will also get captured and if it is the end of the list, then it'll move to next keyword.
The script and tasks are quite simple but ive mentioned in detail in order to completely explain about the issues.
Also ive attached a video of the loading point to explain it a bit better
The scraper has field to enter multiple keywords and it scrapes the page one by one for the mentioned keywords in order. Once a keyword is completed, the entry file automatically gets downloaded and it moves to next search.
But currently while scraping, initially the speed of scraping is good and it scrapes like 20 entries in 2-3 seconds. But after sometime like 30-45 minutes of scraping the speed gradually reduces to like taking 2-3 seconds for 1 entry.
What we think is that maybe due to high amount of entry accumulation in the page, it gets slowed down. So maybe if we clear the entries automatically (which currently isnt done) while moving from one keyword to another, it wont get slowed and the file will be downloaded as it is done for every keyword.
Also, sometimes while scraping, the page gets stuck on loading and the scraper thinks that entries are finished and hence it moves to next keyword.
But I observed that in order to load it further, we need to click on any entry and then click on back button, after which, the page gets loaded further. So we need to add a command, which would make the scraper click on any of the entries and then on back button before moving to the next keyword. So this way if there are more entries, then they will also get captured and if it is the end of the list, then it'll move to next keyword.
The script and tasks are quite simple but ive mentioned in detail in order to completely explain about the issues.
Also ive attached a video of the loading point to explain it a bit better