Data Pipeline Integration for Food Databases
Budget: $10 – $30 USD
Summary
Need a Data Pipeline Engineer (Python / ETL / PostgreSQL)
We are looking for an experienced Data Pipeline Engineer to integrate multiple food databases into an existing production backend.
Important: This is not a greenfield project. The backend, database schema, APIs, and food intelligence engine have already been built. Your role is to build and maintain the ingestion pipelines that feed the existing system.
Current Backend Status
* Existing PostgreSQL database schema
* Existing backend APIs
* Existing food scoring engine
* Existing product model and normalization framework
* Existing infrastructure for product ingestion
Scope of Work
Build reliable ingestion pipelines to import, clean, normalize, validate, and synchronize data from the following sources:
* Open Food Facts
* USDA FoodData Central
* One Latin American food database
* One Chinese food database
* One Southeast Asian food database
Your responsibilities include:
* Downloading data from APIs or bulk datasets
* Parsing and transforming source data
* Mapping source fields into our existing schema
* Deduplicating products
* Importing and linking product images where available
* Building reliable, resumable ETL pipelines
* Creating incremental update jobs
* Producing validation and import reports
* Documenting the ingestion process
Required Skills
* Python
* PostgreSQL
* SQL
* ETL / ELT pipeline development
* Data modeling
* REST APIs
* JSON / CSV / XML processing
* Data validation and deduplication
* Git
* Docker (preferred)
Experience with food datasets, product catalogs, or large-scale data ingestion is a strong plus.
Long-Term Opportunity
This engagement represents the first phase of a much larger data infrastructure initiative. There is a strong possibility of a long-term collaboration as we continue expanding our global food intelligence platform by integrating additional regional and commercial data sources. We are looking for someone who can grow with the project and contribute to future phases.
When Applying
Please include the following in your initial proposal:
1. Your proposed technical approach for integrating these data sources into our existing backend.
2. Relevant ETL or data pipeline projects you have completed.
3. Your proposed implementation timeline, including major milestones.
4. Your proposed fixed-price or milestone-based quote for completing this scope of work.
Please include your proposed timeline and quote in your initial proposal. We are not looking to go back and forth to obtain this information. Applications that do not include a proposed approach, timeline, and quote may not be considered.
We’re looking for someone who can begin immediately and deliver a robust, maintainable pipeline that can be extended with additional data sources in future phases.
Need a Data Pipeline Engineer (Python / ETL / PostgreSQL)
We are looking for an experienced Data Pipeline Engineer to integrate multiple food databases into an existing production backend.
Important: This is not a greenfield project. The backend, database schema, APIs, and food intelligence engine have already been built. Your role is to build and maintain the ingestion pipelines that feed the existing system.
Current Backend Status
* Existing PostgreSQL database schema
* Existing backend APIs
* Existing food scoring engine
* Existing product model and normalization framework
* Existing infrastructure for product ingestion
Scope of Work
Build reliable ingestion pipelines to import, clean, normalize, validate, and synchronize data from the following sources:
* Open Food Facts
* USDA FoodData Central
* One Latin American food database
* One Chinese food database
* One Southeast Asian food database
Your responsibilities include:
* Downloading data from APIs or bulk datasets
* Parsing and transforming source data
* Mapping source fields into our existing schema
* Deduplicating products
* Importing and linking product images where available
* Building reliable, resumable ETL pipelines
* Creating incremental update jobs
* Producing validation and import reports
* Documenting the ingestion process
Required Skills
* Python
* PostgreSQL
* SQL
* ETL / ELT pipeline development
* Data modeling
* REST APIs
* JSON / CSV / XML processing
* Data validation and deduplication
* Git
* Docker (preferred)
Experience with food datasets, product catalogs, or large-scale data ingestion is a strong plus.
Long-Term Opportunity
This engagement represents the first phase of a much larger data infrastructure initiative. There is a strong possibility of a long-term collaboration as we continue expanding our global food intelligence platform by integrating additional regional and commercial data sources. We are looking for someone who can grow with the project and contribute to future phases.
When Applying
Please include the following in your initial proposal:
1. Your proposed technical approach for integrating these data sources into our existing backend.
2. Relevant ETL or data pipeline projects you have completed.
3. Your proposed implementation timeline, including major milestones.
4. Your proposed fixed-price or milestone-based quote for completing this scope of work.
Please include your proposed timeline and quote in your initial proposal. We are not looking to go back and forth to obtain this information. Applications that do not include a proposed approach, timeline, and quote may not be considered.
We’re looking for someone who can begin immediately and deliver a robust, maintainable pipeline that can be extended with additional data sources in future phases.