Media monitoring app
Budget: $250 – $750 USD
Utility application for automating OSINT on news sites, public messenger channels, social media.
Monitoring app that will allow USER to mine (aggregate) a list of links based on search queries. Similar to Google alerts but based on a user’s set list of domains which user specifies and maintains.
Actual scraping and storing of query data is not required (ie. only basic analytics: source link, article date, article name).
The spider will run mostly media, news sites, public messenger channels, ie. Telegram, integrate with Crowdtangle for FB (Meta).
Once initial query is compiled and database is populated, the Spider will update new content and add it to query database.
Key functionality:
• Command and control WEB page (Dashboard)
• Query design
• Domain list management
• Spider Configuration through web interface
• Export scan results to file
• Data-driven, scalable horizontally
• Fast and responsive due to growing database data on queries
• Persistent to avoid bans (legal and source resection)
• Ability to circumvent obstacles related to scraping automation tools
• Ability to handle JS heavy sites
Monitoring app that will allow USER to mine (aggregate) a list of links based on search queries. Similar to Google alerts but based on a user’s set list of domains which user specifies and maintains.
Actual scraping and storing of query data is not required (ie. only basic analytics: source link, article date, article name).
The spider will run mostly media, news sites, public messenger channels, ie. Telegram, integrate with Crowdtangle for FB (Meta).
Once initial query is compiled and database is populated, the Spider will update new content and add it to query database.
Key functionality:
• Command and control WEB page (Dashboard)
• Query design
• Domain list management
• Spider Configuration through web interface
• Export scan results to file
• Data-driven, scalable horizontally
• Fast and responsive due to growing database data on queries
• Persistent to avoid bans (legal and source resection)
• Ability to circumvent obstacles related to scraping automation tools
• Ability to handle JS heavy sites