Dorks and scraping
Budget: €1,500 – €3,000 EUR
I want to do the following Project:
1st Stage (1st Software)
- Create a personalized search engine for a specific list of sites;
- When i Click in a Button a list of keywords are searched in my own Google search engine
(My Google search engine can be done here: https://programmablesearchengine.google.com/about/ )
- This search engine will use specific Dorks to search if these words exist on some of the sites previously selected by me and added in the Profile of my own Google search engine;
- Then the links to the pages of the preselected websites that my Google search engine said that these words exist will be placed in an Excel in Google Drive;
2nd Stage (2nd software)
- Then whenever I click on a button in the software, the software will look for the links in Excel from Google Drive and open these sites/pages;
- There will be many sites (potentially more than 1000 in the short term);
- As there will be many websites, I think they should be opened in a lightweight browser (Lynx or headless);
- Some of these sites required a proxy to open;
- Then whenever I click on another button in the software, I want the software in a certain location on each site/page (using coordinates or by section or another way that the Programmer finds more effective to program) to check if the color is green or red every second (for 30 seconds);
-> After the software collects information on how many seconds the color took to change from green to red from all different websites/pages, it sends this information to another Excel file in Google Drive;
I prefer software to work on Windows.
Note:
- I need the Source Code in case I want another programmer to work from the base I am ordering here;
- I need you to maintain this software for me and have the possibility of making Updates, changes, Bug fixes over time (Hourly price for mantanance to be agreed)
I Belive This tecnologies can Help:
- DCOM (acrônimo para Distributed Component Object Model)
- Node.js
- React.js
- lynx browser
- hedless browser
- Ajax
- Webhooks
- websockets
- REST API
- Web Scraping
1st Stage (1st Software)
- Create a personalized search engine for a specific list of sites;
- When i Click in a Button a list of keywords are searched in my own Google search engine
(My Google search engine can be done here: https://programmablesearchengine.google.com/about/ )
- This search engine will use specific Dorks to search if these words exist on some of the sites previously selected by me and added in the Profile of my own Google search engine;
- Then the links to the pages of the preselected websites that my Google search engine said that these words exist will be placed in an Excel in Google Drive;
2nd Stage (2nd software)
- Then whenever I click on a button in the software, the software will look for the links in Excel from Google Drive and open these sites/pages;
- There will be many sites (potentially more than 1000 in the short term);
- As there will be many websites, I think they should be opened in a lightweight browser (Lynx or headless);
- Some of these sites required a proxy to open;
- Then whenever I click on another button in the software, I want the software in a certain location on each site/page (using coordinates or by section or another way that the Programmer finds more effective to program) to check if the color is green or red every second (for 30 seconds);
-> After the software collects information on how many seconds the color took to change from green to red from all different websites/pages, it sends this information to another Excel file in Google Drive;
I prefer software to work on Windows.
Note:
- I need the Source Code in case I want another programmer to work from the base I am ordering here;
- I need you to maintain this software for me and have the possibility of making Updates, changes, Bug fixes over time (Hourly price for mantanance to be agreed)
I Belive This tecnologies can Help:
- DCOM (acrônimo para Distributed Component Object Model)
- Node.js
- React.js
- lynx browser
- hedless browser
- Ajax
- Webhooks
- websockets
- REST API
- Web Scraping