MVP for Website Monitoring/Audit Suite

Full-Stack Developer for MVP Website Monitoring and Audit Suite
We are seeking a skilled Full-Stack Developer to create a Minimum Viable Product (MVP) for a website monitoring and audit suite. The ideal candidate will have experience with both front-end and back-end technologies to build a user-friendly and efficient platform. You will be responsible for designing, developing, and implementing features that allow users to monitor website performance and conduct audits effectively. Strong problem-solving skills and attention to detail are essential. If you are passionate about building innovative solutions, we would love to hear from you!
Key Responsibilities:
Scanner Engine: Build a resilient microservice using Puppeteer and the Lighthouse Engine capable of mobile-first emulation and hardware throttling.
Time-Series Data: Implement PostgreSQL with the TimescaleDB extension to manage millions of historical performance metrics via Hypertables.
Job Queuing: Manage background scanning tasks using Redis and BullMQ to ensure horizontal scalability.
API Design: Develop RESTful endpoints for on-demand scans, time-series data retrieval, and automated white-label PDF generation.

Technical Requirements:
Node.js / TypeScript Mastery (NestJS or Express preferred).
Expertise in Puppeteer/Playwright and bypassing WAFs (Stealth plugins/Residential Proxies).
Strong experience with PostgreSQL and time-series extensions (TimescaleDB/InfluxDB).
Experience with Next.js for a performant frontend dashboard.
Deliverable: A fully functioning MVP capable of handling 5,000+ daily scans across 20+ global nodes.

Milestone Payment Schedule (8 - Week MVP)
Milestone 1: Data Foundation & Infrastructure (20%)

Focus: Establishing a high-performance time-series database and the "Scraper" core.

Week 1 (Infra & DB):

Set up Dockerized environments (Production/Staging).

Initialize PostgreSQL and enable the TimescaleDB extension.

Task: Create the scans table and convert it into a hypertable partitioned by a created_at timestamp.

Task: Create a composite index on (user_id, created_at DESC) for fast user-specific history lookups.

Week 2 (Scanner Core):

Develop the Puppeteer script with stealth plugins to avoid detection.

Task: Implement a data-mapping function that converts raw Puppeteer output into the SQL schema.

Deliverable: A script that can be executed via CLI and saves a record successfully.

Milestone 2: Background Processing & Scalability (30%)

Focus: Moving from a manual script to a production-grade asynchronous queue.

Week 3 (Queue Setup):

Deploy Redis for state management.

Set up BullMQ with a dedicated scanner-worker process.

Task: Implement "Global Concurrency Limiting" (e.g., max 5 Puppeteer instances) to prevent server RAM exhaustion.

Week 4 (API Development):

Endpoint: POST /v1/scan – Accepts a URL, validates it, and pushes a job to BullMQ. Returns a jobId.

Endpoint: GET /v1/scan/:jobId – Checks Redis to return pending, active, failed, or completed.

Task: Implement Exponential Backoff logic (retrying a scan if it hits a 403 or timeout error).

Milestone 3: UI & Analytics Engine (30%)

Focus: Speeding up data retrieval and building the user interface.

Week 5 (Data Aggregation):

Task: Create TimescaleDB Continuous Aggregates (Materialized Views) for daily/weekly averages so the frontend doesn't have to calculate thousands of rows on every refresh.

Task: Set up a Refresh Policy to update these aggregates every hour automatically.

Week 6 (Frontend Integration):

Endpoint: GET /v1/analytics/history – Returns aggregated data from the Continuous Aggregate.

Task: Connect the UI to a WebSocket (using Socket.io) to provide real-time updates on the progress bar while a scan is running.

Deliverable: A dashboard that loads in under 500ms even with 10,000+ data points.

Milestone 4: Billing, Reporting & Hardening (20%)

Focus: Monetization and production stability.

Week 7 (Monetization & Reporting):

Endpoint: POST /v1/billing/create-subscription – Integrates Stripe Checkouts.

Task: Implement a Stripe Webhook listener to enable/disable user features based on payment status.

Task: Use a library like jspdf or puppeteer-pdf to generate a downloadable report from the dashboard view.

Week 8 (Testing & Launch):

Task: Conduct Load Testing—simulate 50 users triggering scans simultaneously to verify Redis/BullMQ stability.

Task: Final security audit (Environment variable protection, rate-limiting on APIs).

Deliverable: Production deployment on your final server (AWS/DigitalOcean/Hetzner).

Please find the attached requirements doc.