MERN Stack ETL Engineer Needed
Budget: $30 – $250 USD
I’m looking for someone who can put together a lightweight but reliable ETL workflow on the MERN stack. My end-goal is fully automated data population and synchronization, so that new and updated records coming from several sources flow cleanly into MongoDB with no manual touch-ups. This is for a web app mediating between sellers and buyers of services/products, focusing on company data population to accelerate user onboarding.
Sources you’ll be working with:
• REST-based APIs (e.g., Crunchbase, Google Business for company details)
• Existing SQL/NoSQL databases (public datasets or internal ones)
• Periodic CSV / Excel drops (from external APIs or manual uploads)
• A small text-only web-scraping feed (ethical scraping from public sites, no heavy crawling)
Core tasks:
Build the extraction layer in Node.js so each source can be polled or triggered on a schedule using tools like cron or n8n for automation.
Apply basic transforms—type casting, deduplication (using fuzzy matching like Fuse.js), simple enrichments (e.g., geocoding via Google Maps API)—and log any rows that fail validation (using libraries like Yup or Joi).
Load the cleaned data into MongoDB collections, keeping schemas consistent with the current React front-end. Handle upserts for updates and use bulk operations for performance.
Expose a status endpoint in Express that confirms last run time, rows processed, and any errors. Include basic monitoring with tools like PM2.
Package scripts so they can be deployed via a single npm run command and documented in a concise README. Ensure compatibility with Docker for easy deployment.
I have access credentials and small sample datasets ready; you’ll get them on kickoff. If you’ve built Node-based pipelines before, especially with Axios, Cheerio, or similar scraping tools, this should feel familiar. Clean code, clear logging, and brief setup notes are the only formal acceptance criteria—I’ll handle UI work on my side once the data is flowing. Preferred skills: Experience with ETL tools (n8n/Airbyte), data validation, and handling large datasets. Estimated time: 10-20 hours, depending on complexity.
Sources you’ll be working with:
• REST-based APIs (e.g., Crunchbase, Google Business for company details)
• Existing SQL/NoSQL databases (public datasets or internal ones)
• Periodic CSV / Excel drops (from external APIs or manual uploads)
• A small text-only web-scraping feed (ethical scraping from public sites, no heavy crawling)
Core tasks:
Build the extraction layer in Node.js so each source can be polled or triggered on a schedule using tools like cron or n8n for automation.
Apply basic transforms—type casting, deduplication (using fuzzy matching like Fuse.js), simple enrichments (e.g., geocoding via Google Maps API)—and log any rows that fail validation (using libraries like Yup or Joi).
Load the cleaned data into MongoDB collections, keeping schemas consistent with the current React front-end. Handle upserts for updates and use bulk operations for performance.
Expose a status endpoint in Express that confirms last run time, rows processed, and any errors. Include basic monitoring with tools like PM2.
Package scripts so they can be deployed via a single npm run command and documented in a concise README. Ensure compatibility with Docker for easy deployment.
I have access credentials and small sample datasets ready; you’ll get them on kickoff. If you’ve built Node-based pipelines before, especially with Axios, Cheerio, or similar scraping tools, this should feel familiar. Clean code, clear logging, and brief setup notes are the only formal acceptance criteria—I’ll handle UI work on my side once the data is flowing. Preferred skills: Experience with ETL tools (n8n/Airbyte), data validation, and handling large datasets. Estimated time: 10-20 hours, depending on complexity.
Related categories:
JavaScript
NoSQL Couch & Mongo
Hadoop
Node.js
Elasticsearch
MongoDB
ETL
MERN Stack
MERN
REST API