OSINT automation web app
Budget: $250 – $750 USD
Web app, utility application for automating OSINT.
Monitoring bot that will allow groups of users to mine (aggregate) a list of Web/Telegram messenger links based on keyword mentions (queries). Similar to Google alerts but based on a user’s set list of web domains/Telegram channels which user specifies and maintains. Key objective is to identify sources based on query, bot will return source is query is successful. Detailed scraping of actual data is not required (ie. only basic analytics, article date, article name).
The bot (spider) will run mostly media, news, aggregator sites and public telegram channels.
Once initial query is compiled and database is populated, the bot will alert once keyword is mentioned in new content and add it to query database. Scraping or saving of the actual content is not required.
Key functionality:
Landing page with authorization
Query design
Bot domain list
Bot Configuration through web interface
Content mining, scraping
Bot Command and control
Data analytics and visualization
Managing, administrating users and groups
Alert options: by email
Flag, record maintain incidents
Data-driven, scalable horizontally
Fast and responsive due to growing database data on queries
Persistent to avoid bans (legal and source resection)
Ability to circumvent obstacles related to scraping automation tools
Ability to handle JS heavy sites
Scheduling options. It’s desirable to have an option of response time on query update in under 5 minutes subject to user configuration.
Monitoring bot that will allow groups of users to mine (aggregate) a list of Web/Telegram messenger links based on keyword mentions (queries). Similar to Google alerts but based on a user’s set list of web domains/Telegram channels which user specifies and maintains. Key objective is to identify sources based on query, bot will return source is query is successful. Detailed scraping of actual data is not required (ie. only basic analytics, article date, article name).
The bot (spider) will run mostly media, news, aggregator sites and public telegram channels.
Once initial query is compiled and database is populated, the bot will alert once keyword is mentioned in new content and add it to query database. Scraping or saving of the actual content is not required.
Key functionality:
Landing page with authorization
Query design
Bot domain list
Bot Configuration through web interface
Content mining, scraping
Bot Command and control
Data analytics and visualization
Managing, administrating users and groups
Alert options: by email
Flag, record maintain incidents
Data-driven, scalable horizontally
Fast and responsive due to growing database data on queries
Persistent to avoid bans (legal and source resection)
Ability to circumvent obstacles related to scraping automation tools
Ability to handle JS heavy sites
Scheduling options. It’s desirable to have an option of response time on query update in under 5 minutes subject to user configuration.