BOT: social media monitoring
Budget: $250 – $750 USD
Utility application for automating OSINT on news sites, public messenger channels, social media.
Monitoring bot that will allow USER to mine (aggregate) a list of links based on search queries. Similar to Google alerts but based on a user’s set list of domains which user specifies and maintains. Actual scraping and storing of query data is not required (ie. only basic analytics: source link, article date, article name).
The bot (spider) will run mostly media, news sites, public messenger channels, ie. Telegram, integrate with Crowdtangle for FB (Meta). Once initial query is compiled and database is populated, the bot will update new content and add it to query database.
Key functionality:
• Bot Command and control WEB page (Dashboard)
• Query design
• Domain list management
• Bot Configuration through web interface
• Export scan results to file
• Data-driven, scalable horizontally
• Fast and responsive due to growing database data on queries
• Persistent to avoid bans (legal and source resection)
• Ability to circumvent obstacles related to scraping automation tools
• Ability to handle JS heavy sites
Monitoring bot that will allow USER to mine (aggregate) a list of links based on search queries. Similar to Google alerts but based on a user’s set list of domains which user specifies and maintains. Actual scraping and storing of query data is not required (ie. only basic analytics: source link, article date, article name).
The bot (spider) will run mostly media, news sites, public messenger channels, ie. Telegram, integrate with Crowdtangle for FB (Meta). Once initial query is compiled and database is populated, the bot will update new content and add it to query database.
Key functionality:
• Bot Command and control WEB page (Dashboard)
• Query design
• Domain list management
• Bot Configuration through web interface
• Export scan results to file
• Data-driven, scalable horizontally
• Fast and responsive due to growing database data on queries
• Persistent to avoid bans (legal and source resection)
• Ability to circumvent obstacles related to scraping automation tools
• Ability to handle JS heavy sites