Seeking way to scrape names from URL into database
Budget: $750 – $1,500 USD
We are looking for a system which can do the following:
1) Site with a field in which to add a URL Link (e.g., https://b.tc/conference/speakers)
2) The engine analyzes each name and repopulates into a table -- showing not only the Twitter but also any other social media information the system could find on individuals
3) The page would then show a table with all names listed in the speaker list and a button by each name that says "add to Database"
The algorithm would need to do the following process automatically:
1. Recognize on the page and list first name / last name
2. Record identifiable information from that link (e.g., bio image, geography, title, company, keyword--e.g., bitcoin)
3. Google search the name and keyword with most recent as of, say, 2 years ago
4. Search top 10 search results and confirm name mentioned in Google search result is the same person sought (by searching for identifiable information)
5. Add links to database associated with that individual
6. Populate links in table presented to user (link to a database in our system)
7. User can click on button to 'rerun' for those individuals without any links (in which case, system searches the next 10 pages, etc.--- or whatever threshold of pages makes sense OR the system reruns the search algorithm until there are at minimum 5 links - iterative search up to 10 times, say
8. Of those who are 'added' they show up in the user's account.
9. The initial link where the individual was found on would be populated as a field as well
10. A summary table would need to be generated (e.g., 4 links, 238 professionals, 148 added to database, 21% overlap [with people and URLs added - say, if people are adding multiple speaker lists from conferences) etc.,
This example is just for speaker lists but could, of course, work anywhere where the URL has names listed on the page using NLP (?) which recognizes first name last name. Press releases would be another great one, or Fortune article, etc.
1) Site with a field in which to add a URL Link (e.g., https://b.tc/conference/speakers)
2) The engine analyzes each name and repopulates into a table -- showing not only the Twitter but also any other social media information the system could find on individuals
3) The page would then show a table with all names listed in the speaker list and a button by each name that says "add to Database"
The algorithm would need to do the following process automatically:
1. Recognize on the page and list first name / last name
2. Record identifiable information from that link (e.g., bio image, geography, title, company, keyword--e.g., bitcoin)
3. Google search the name and keyword with most recent as of, say, 2 years ago
4. Search top 10 search results and confirm name mentioned in Google search result is the same person sought (by searching for identifiable information)
5. Add links to database associated with that individual
6. Populate links in table presented to user (link to a database in our system)
7. User can click on button to 'rerun' for those individuals without any links (in which case, system searches the next 10 pages, etc.--- or whatever threshold of pages makes sense OR the system reruns the search algorithm until there are at minimum 5 links - iterative search up to 10 times, say
8. Of those who are 'added' they show up in the user's account.
9. The initial link where the individual was found on would be populated as a field as well
10. A summary table would need to be generated (e.g., 4 links, 238 professionals, 148 added to database, 21% overlap [with people and URLs added - say, if people are adding multiple speaker lists from conferences) etc.,
This example is just for speaker lists but could, of course, work anywhere where the URL has names listed on the page using NLP (?) which recognizes first name last name. Press releases would be another great one, or Fortune article, etc.