Web Scraper & Subscription Site Development

Project Title: Public Tender Scraper & Subscription Website (Trinidad & Tobago + Guyana)

Project Description:
We need an experienced developer (or team) to build a web scraping and aggregation platform that collects public tender data from all official government and related websites in Trinidad and Tobago and Guyana. The scraped tender information will be consolidated into a single, searchable website with subscription-based access.

Key Requirements:

Research and identify all online sources of public tender information from both governments.

Develop robust scraping modules that handle various data formats (HTML, PDFs, Excel) and dynamically rendered content.

Store data efficiently in a structured database with historical records.

Build a backend API to serve tender data and manage user subscriptions.

Create a responsive frontend website for users to search, filter, and view tender details.

Integrate subscription and payment systems (e.g., Stripe).

Design the system to scale for future expansion to other countries.

Follow legal and ethical scraping practices.

Provide comprehensive documentation.


Deliverables:

Fully functional scrapers for both countries.

Centralized tender database.

Subscription-based website with user account management.

UX dashboard prototype/demo showing key user workflows and admin monitoring.

Documentation for deployment and maintenance.


Preferred Skills:

Web scraping (Scrapy, Puppeteer, Selenium, etc.)

Backend development (Python/Django/Flask/FastAPI or Node.js/Express)

Frontend development (React, Vue, Angular)

Database design (PostgreSQL, MongoDB)

API development

Payment gateway integration (Stripe, PayPal)

DevOps experience (Docker, CI/CD)


Please include examples of previous scraping or subscription platform projects in your past.