Web Scraping AI Tool Directories

I want a full scrape of every tool listed on the ten AI-directory sites below so I can analyse the landscape in one consolidated dataset:

Futurepedia.io
Toolify.ai
There’s An AI For That
ProductHunt (AI category only)
AIxploria
TopAI.tools
TheAISurf / AIChief
Toolpilot.ai
Insidr.ai
AI Mojo

For each tool I will at minimum need its name, a short description and the canonical URL. If the page also exposes pricing details, ratings, reviews or any other structured fields, please capture those too; the richer the dataset, the better.

I’m flexible on output—CSV, JSON or an Excel sheet all work for me—so feel free to choose the most convenient format in your pipeline and we can convert later if needed. I would also like the underlying Python (or similar) script so I can rerun the extraction whenever the directories update.

Please respect each site’s robots.txt and usage terms, and code the scraper with polite request intervals to avoid rate-limiting. Once you have an initial sample ready, share it so I can confirm the field structure before you harvest the full data.