Data Scrape & MongoDB Migration
Budget: $30 – $250 USD
My React + Node application currently runs on a single SQLite file and everything works exactly as I want. Now I’m ready to extend it with two small but important upgrades:
1. Scrape content from several specific websites I will provide and store that content in a brand-new database table/collection. The scraper can be written in Node (Puppeteer, Cheerio, Axios—whatever you prefer) as long as it runs headless on my VPS and can be triggered by a simple npm script or cron.
2. Convert the existing SQLite structure to MongoDB so the new scraped data and all current records live in the same database. This is a straightforward migration: replicate the current schema, migrate the data, then update the existing Node data-access layer so the rest of the app keeps behaving exactly as it does now.
Acceptance criteria
• A repeatable scraper script pulling data from the specified sites and inserting it into MongoDB without duplicates
• All existing API endpoints continue to work after the switch from SQLite to MongoDB
• A brief readme explaining how to run the scraper and how the migration was performed
The core codebase is clean, well-structured, and under Git; I will give you access to a feature branch so we can keep things isolated until we merge.
1. Scrape content from several specific websites I will provide and store that content in a brand-new database table/collection. The scraper can be written in Node (Puppeteer, Cheerio, Axios—whatever you prefer) as long as it runs headless on my VPS and can be triggered by a simple npm script or cron.
2. Convert the existing SQLite structure to MongoDB so the new scraped data and all current records live in the same database. This is a straightforward migration: replicate the current schema, migrate the data, then update the existing Node data-access layer so the rest of the app keeps behaving exactly as it does now.
Acceptance criteria
• A repeatable scraper script pulling data from the specified sites and inserting it into MongoDB without duplicates
• All existing API endpoints continue to work after the switch from SQLite to MongoDB
• A brief readme explaining how to run the scraper and how the migration was performed
The core codebase is clean, well-structured, and under Git; I will give you access to a feature branch so we can keep things isolated until we merge.