Mac Software Executable - Website Domain Crawler
Budget: $30 – $250 USD
I am looking for a Mac Software that will run a bot that will crawl all .com.au domains and check for the software used behind the Website such as Wappalyzer use. Wordpress / Wix / Square Space /
Then add Icon of Caution mark if finds this
"There has been a critical error on this website.
Learn more about troubleshooting WordPress."
Or some major errors on site.
The crawler will work once every XX minutes that I set, going from domain to domain. Default is 1 minute.
I will run the DMG file on my Mac and watch the UI crawl each domain and save the info with YES / NO if broken WP . And show caution icon if Yes.
Allow export to Excel file button.
Settings will be
Time between Crawls
Check for "WP Broken" YES/ NO
Auto Start Time / Auto Finish Time for example 9am to 5pm each day.
Then add Icon of Caution mark if finds this
"There has been a critical error on this website.
Learn more about troubleshooting WordPress."
Or some major errors on site.
The crawler will work once every XX minutes that I set, going from domain to domain. Default is 1 minute.
I will run the DMG file on my Mac and watch the UI crawl each domain and save the info with YES / NO if broken WP . And show caution icon if Yes.
Allow export to Excel file button.
Settings will be
Time between Crawls
Check for "WP Broken" YES/ NO
Auto Start Time / Auto Finish Time for example 9am to 5pm each day.
Related categories:
Business, Accounting, Human Resources & Legal
Python
WordPress
Software Architecture
Mac OS