Specialized Search Engine and AI Creation, ETL pipeline management

I'm looking for a Senior Backend/AI Engineer to create a specialized search engine system using Python, PostgreSQL, OpenAI API, and other advanced technologies. The role involves building a robust ETL pipeline, implementing semantic search, and ensuring strict quality assurance.

Scope of work
- Build a robust ETL pipeline in Python to process 38k+ categories and company data with Hash Checking for incremental updates
- Implement Semantic Search using Hybrid Search (Vector + BM25)
- Ensure the system passes predefined 'Blind Test' protocol
- Utilize strict type hinting and deterministic logic for code architecture
- Deliver production-grade, containerized code
- Project management tracking in Jira, with documentation and testing included

Data engineering expertise
ETL, Data validation, SQL

Data technology tools
Python, SQL

Additional information
The project requires strict quality assurance and predefined testing protocols. Please provide a fixed price quote and timeline for completion.


Ideal Candidate
- Fluent in SQL and Vector Embeddings
- Experience with Atomic Swaps in databases is a huge plus
- Must deliver production-grade, containerized code

Delivery time: 5 weeks
Budget: around 2000 USD