Bulk Vehicle Data Import Developer
Budget: $30 – $250 USD
Job Title: NestJS Backend Developer – High-Performance Car Bulk Import (ETL)
The Challenge
We are looking for a senior-level NestJS developer to build a robust, production-ready car data import engine for our vehicle marketplace. This isn't a simple "upload and save" task; we require a sophisticated streaming pipeline capable of processing massive datasets (CSV, XML, JSON) with minimal memory footprint and high reliability.
Core Task
Build a POST /imports/cars endpoint that:
Automatic Format Detection: Handles multipart/form-data and identifies the file type (CSV, XML, or JSON) programmatically.
Stream-Based Processing: Processes data using Node.js Streams / AsyncIterables. The application should never load the full file into RAM.
Data Pipeline: Implements a clean Parse → Validate/Normalize → Map → Persist flow.
Smart Idempotency: Ensures no duplicate vehicle records are created on re-imports (e.g., via unique VIN hashing or business-key indexing).
Atomic Feedback: Returns a structured summary (total processed, success count, and a detailed array of failed rows with specific validation errors).
Technical Stack & Requirements
Framework: NestJS (TypeScript in Strict Mode).
Architecture: Clean, SOLID principles; logic must reside in dedicated Services/Providers, keeping Controllers thin.
Validation: Heavy usage of DTOs with class-validator. We are open to Zod for complex runtime schema enforcement within the stream.
Streaming Tools: * CSV: csv-parser or fast-csv.
XML: fast-xml-parser (streaming mode).
JSON: stream-json.
Database: MongoDB via Mongoose.
Reliability: Proper backpressure handling and error boundaries to ensure one corrupt row doesn't crash the entire import process.
Performance & Quality (Nice-to-Have)
Progress Tracking: Experience with WebSockets (Socket.io) or Redis-based status tracking for long-running imports.
Testing: High coverage with Jest (specifically unit tests for the mapping logic and integration tests for the stream).
Cloud: Experience staging large files in AWS S3 before processing.
What We Offer
A highly focused, well-defined technical task.
Clear acceptance criteria and a "no-fluff" development environment.
Potential for ongoing collaboration on our core platform.
How to Apply
We value brevity and technical depth. Please provide:
Relevant Experience: A 3-sentence summary of the largest/most complex import system you’ve built.
Code Samples: A link to a GitHub repo or Gist showing a NestJS service or a streaming implementation.
The "How-To": Briefly outline your strategy for handling backpressure and how you would structure the error-reporting object for a 100,000-row file.
Note: Only applicants with demonstrated experience in NestJS and Node.js Streams will be considered.
The Challenge
We are looking for a senior-level NestJS developer to build a robust, production-ready car data import engine for our vehicle marketplace. This isn't a simple "upload and save" task; we require a sophisticated streaming pipeline capable of processing massive datasets (CSV, XML, JSON) with minimal memory footprint and high reliability.
Core Task
Build a POST /imports/cars endpoint that:
Automatic Format Detection: Handles multipart/form-data and identifies the file type (CSV, XML, or JSON) programmatically.
Stream-Based Processing: Processes data using Node.js Streams / AsyncIterables. The application should never load the full file into RAM.
Data Pipeline: Implements a clean Parse → Validate/Normalize → Map → Persist flow.
Smart Idempotency: Ensures no duplicate vehicle records are created on re-imports (e.g., via unique VIN hashing or business-key indexing).
Atomic Feedback: Returns a structured summary (total processed, success count, and a detailed array of failed rows with specific validation errors).
Technical Stack & Requirements
Framework: NestJS (TypeScript in Strict Mode).
Architecture: Clean, SOLID principles; logic must reside in dedicated Services/Providers, keeping Controllers thin.
Validation: Heavy usage of DTOs with class-validator. We are open to Zod for complex runtime schema enforcement within the stream.
Streaming Tools: * CSV: csv-parser or fast-csv.
XML: fast-xml-parser (streaming mode).
JSON: stream-json.
Database: MongoDB via Mongoose.
Reliability: Proper backpressure handling and error boundaries to ensure one corrupt row doesn't crash the entire import process.
Performance & Quality (Nice-to-Have)
Progress Tracking: Experience with WebSockets (Socket.io) or Redis-based status tracking for long-running imports.
Testing: High coverage with Jest (specifically unit tests for the mapping logic and integration tests for the stream).
Cloud: Experience staging large files in AWS S3 before processing.
What We Offer
A highly focused, well-defined technical task.
Clear acceptance criteria and a "no-fluff" development environment.
Potential for ongoing collaboration on our core platform.
How to Apply
We value brevity and technical depth. Please provide:
Relevant Experience: A 3-sentence summary of the largest/most complex import system you’ve built.
Code Samples: A link to a GitHub repo or Gist showing a NestJS service or a streaming implementation.
The "How-To": Briefly outline your strategy for handling backpressure and how you would structure the error-reporting object for a 100,000-row file.
Note: Only applicants with demonstrated experience in NestJS and Node.js Streams will be considered.