Automated Area-Guide Comparison Bot
Budget: ₹1,500 – ₹12,500 INR
I need a small, reliable bot that crawls the area-guide pages on propertyfinder.ae and bayut every 24 hours, beginning with Dubai Marina and expanding easily to any neighbourhood I add later. The goal is to tell me—at a glance—where propertyfinder’s guides lag behind the competition so my content team knows exactly what to fix next.
What the bot must capture
• Content structure and layout: pull every section header, keep the heading hierarchy (H1–H6), note order, paragraph length and any calls-to-action above the fold.
• Information depth and quality: calculate word counts, Flesch readability, and flag missing or thin sections.
• Visual elements: log each image, map, video or infographic, plus the alt text length.
• Navigation & overall UX: record presence of sticky menus, breadcrumbs, jump links and internal links that aid exploration.
Daily report
The script should output a side-by-side comparison (CSV or Google Sheet) plus a short narrative summary emailed or pushed to Slack. I want the report to spotlight section headers and organisation, text formatting and readability, and navigation/user-experience issues for every area guide scanned.
Technical expectations
• Clean, documented Python (BeautifulSoup + Selenium/Playwright, pandas) or Node.js—your choice—as long as it runs headless, rate-limits requests and respects robots.txt.
• Single-command setup on my Mac or a small Ubuntu droplet; include clear instructions to add new areas or tweak selectors when the sites change.
• Cron-friendly so it refreshes automatically each day.
Acceptance criteria
– First run compares the Dubai Marina guides and surfaces at least five structural differences.
– Output columns: page URL, crawl date, section title, word count, readability score, visual-element flags, CTA-above-fold (yes/no).
– Email/Slack notification lands without manual intervention.
If this sounds straightforward to you and you can deliver a maintainable, extensible solution, I’m ready to get started right away.
What the bot must capture
• Content structure and layout: pull every section header, keep the heading hierarchy (H1–H6), note order, paragraph length and any calls-to-action above the fold.
• Information depth and quality: calculate word counts, Flesch readability, and flag missing or thin sections.
• Visual elements: log each image, map, video or infographic, plus the alt text length.
• Navigation & overall UX: record presence of sticky menus, breadcrumbs, jump links and internal links that aid exploration.
Daily report
The script should output a side-by-side comparison (CSV or Google Sheet) plus a short narrative summary emailed or pushed to Slack. I want the report to spotlight section headers and organisation, text formatting and readability, and navigation/user-experience issues for every area guide scanned.
Technical expectations
• Clean, documented Python (BeautifulSoup + Selenium/Playwright, pandas) or Node.js—your choice—as long as it runs headless, rate-limits requests and respects robots.txt.
• Single-command setup on my Mac or a small Ubuntu droplet; include clear instructions to add new areas or tweak selectors when the sites change.
• Cron-friendly so it refreshes automatically each day.
Acceptance criteria
– First run compares the Dubai Marina guides and surfaces at least five structural differences.
– Output columns: page URL, crawl date, section title, word count, readability score, visual-element flags, CTA-above-fold (yes/no).
– Email/Slack notification lands without manual intervention.
If this sounds straightforward to you and you can deliver a maintainable, extensible solution, I’m ready to get started right away.