Search Relevance & AI Retrieval Optimization
Budget: $15 – $25 USD
AI-powered search engine that connects recruiters, founders, and investors with the right people faster.
We combine data from LinkedIn, Gmail, and professional networks into a private AI search layer that understands relationships, intent, and relevance.
Our MVP is live with:
• 10K+ candidates embedded in a pgVector database
• TypeScript backend and React frontend
• OpenAI embeddings for candidate matching
Design an evaluation framework to measure search relevance
• Benchmark and experiment with different embedding models (OpenAI, Cohere, SBERT, etc.)
• Implement hybrid retrieval (vector + BM25 + re-ranking)
• Develop and test fusion strategies (RRF, weighted, or learning-to-rank)
• Introduce re-ranking with cross-encoders for top results
• Optimize for both precision and recall with latency under 2 seconds
• Document findings, metrics, and model decisions
• (Later) Help build continuous monitoring and drift detection
⸻
Tech Stack You enjoy
• Python (FastAPI, PyTorch, SentenceTransformers)
• PostgreSQL + pgVector
• OpenAI / Cohere APIs
• rank-bm25 or Elasticsearch
• ranx / ir-measures (evaluation)
• Jupyter, MLflow, or Weights & Biases for experiment tracking
• Integration with existing TypeScript backend (via API)
Test task(paid)
Evaluate and improve the semantic search quality of Yena’s current setup using a small sample dataset.
What to Do
1. Dataset Setup
• Use a sample CSV of 100 candidate profiles (we’ll provide, or you can mock).
• Each profile includes: title, company, location, skills, experience summary.
2. Build Evaluation Sandbox
• Use OpenAI or any embedding model to encode candidates.
• Create 5–10 realistic search queries (e.g. “Senior Python Engineer in Berlin”).
• Manually label 2–3 relevant candidates per query.
3. Compute Search Quality Metrics
• Implement Recall@10 and MRR.
• Output baseline numbers in JSON/CSV.
4. Suggest or Test One Improvement
• Could be model change, BM25 hybridization, or re-ranking.
• Compare metrics and describe what changed.
5. Deliverables
• Jupyter notebook or Python script
• CSV/JSON with metrics results
• 1-page written summary: findings, reasoning, next steps
We combine data from LinkedIn, Gmail, and professional networks into a private AI search layer that understands relationships, intent, and relevance.
Our MVP is live with:
• 10K+ candidates embedded in a pgVector database
• TypeScript backend and React frontend
• OpenAI embeddings for candidate matching
Design an evaluation framework to measure search relevance
• Benchmark and experiment with different embedding models (OpenAI, Cohere, SBERT, etc.)
• Implement hybrid retrieval (vector + BM25 + re-ranking)
• Develop and test fusion strategies (RRF, weighted, or learning-to-rank)
• Introduce re-ranking with cross-encoders for top results
• Optimize for both precision and recall with latency under 2 seconds
• Document findings, metrics, and model decisions
• (Later) Help build continuous monitoring and drift detection
⸻
Tech Stack You enjoy
• Python (FastAPI, PyTorch, SentenceTransformers)
• PostgreSQL + pgVector
• OpenAI / Cohere APIs
• rank-bm25 or Elasticsearch
• ranx / ir-measures (evaluation)
• Jupyter, MLflow, or Weights & Biases for experiment tracking
• Integration with existing TypeScript backend (via API)
Test task(paid)
Evaluate and improve the semantic search quality of Yena’s current setup using a small sample dataset.
What to Do
1. Dataset Setup
• Use a sample CSV of 100 candidate profiles (we’ll provide, or you can mock).
• Each profile includes: title, company, location, skills, experience summary.
2. Build Evaluation Sandbox
• Use OpenAI or any embedding model to encode candidates.
• Create 5–10 realistic search queries (e.g. “Senior Python Engineer in Berlin”).
• Manually label 2–3 relevant candidates per query.
3. Compute Search Quality Metrics
• Implement Recall@10 and MRR.
• Output baseline numbers in JSON/CSV.
4. Suggest or Test One Improvement
• Could be model change, BM25 hybridization, or re-ranking.
• Compare metrics and describe what changed.
5. Deliverables
• Jupyter notebook or Python script
• CSV/JSON with metrics results
• 1-page written summary: findings, reasoning, next steps