Batch Ingestion Pipeline Development

Backend Engineer – Batch Ingestion Pipeline (CSV → Database) We are building a large-scale book content platform.

We are NOT looking for a full-stack developer, UI, or website.

We need a backend engineer to build a simple, reliable batch ingestion pipeline.

SCOPE

Your task is to build a CLI-based ingestion script that:

Reads a CSV file with book metadata

Inserts records into a PostgreSQL database (Supabase)

Handles 10,000–20,000 records efficiently

Uses batch inserts (no per-row inserts)

Runs outside HTTP (terminal / cron / worker)

No frontend.
No dashboard.
No UI.

TECH STACK (Flexible)

Node.js or Python

PostgreSQL

CSV input

Environment variables for DB connection

OUT OF SCOPE

Real-time processing

Web UI

File upload interfaces

EPUB/PDF conversion

SEO, frontend, or design

DELIVERABLES

Clean, documented script

Instructions to run locally or on a server

Example CSV format

Proof that batch insertion works at scale

IDEAL PROFILE

Experience with data pipelines or batch jobs

Comfortable handling large datasets

Understands database performance

Focused on reliability, not UI