Hybrid RAG System Enhancement Project
Budget: $10 – $30 USD
Project Proposal
Title:Enhancement of a Hybrid Retrieval-Augmented Generation System with Keyword and Semantic Search, Reward-Shaped Retrieval, and ReComp-Based Context Compression. [keyvalue](https://www.keyvalue.systems/blog/hybrid-rag-architecture/)
This project proposes a hybrid RAG architecture that combines keyword-based retrieval with semantic search, adds a reward-and-punishment scheme to improve retrieval relevance, and uses ReComp-style context compression to handle long contexts more efficiently. [arxiv](https://arxiv.org/html/2409.13385v1)
Background
Retrieval-Augmented Generation is widely used to reduce LLM limitations such as knowledge cut-off, hallucinations, and lack of transparency, because it grounds generation in retrieved external evidence. Hybrid search is especially useful because keyword retrieval improves exact-match precision while semantic retrieval helps recover conceptually related content. [toloka](https://toloka.ai/blog/rag-evaluation-a-technical-guide-to-measuring-retrieval-augmented-generation/)
Context compression methods such as ReComp-like approaches are important because they reduce prompt size while trying to preserve faithful information from retrieved documents. Recent work also shows that retrieval systems can benefit from reward-based reasoning and reflection mechanisms that explicitly encourage sufficient and accurate evidence use. [arxiv](https://arxiv.org/html/2507.22716v1)
Problem Statement
Current RAG systems often retrieve partially relevant or unrelated passages, and this harms answer quality even when the generation model is strong. The challenge is not only to increase recall, but also to score retrieved items so that related content is rewarded and unrelated content is penalized, especially in the semantic retrieval stage. [superlinked](https://superlinked.com/vectorhub/articles/optimizing-rag-with-hybrid-search-reranking)
Objectives
- Build a hybrid retriever that combines keyword search and semantic search into one ranking pipeline. [keyvalue](https://www.keyvalue.systems/blog/hybrid-rag-architecture/)
- Design a semantic relevance scoring mechanism that rewards closely related passages and penalizes unrelated passages. [arxiv](https://arxiv.org/html/2507.22716v1)
- Add ReComp-based long-context compression so that retrieved evidence can be condensed without losing faithfulness. [iclr](https://iclr.cc/virtual/2024/poster/17885)
- Evaluate the system with a target of above 90% accuracy on general context tasks. [arxiv](https://arxiv.org/html/2504.14891v1)
- Test performance separately across architecture, law, medicine, finance, and computer-related domains, and record the results by domain. [arxiv](https://arxiv.org/html/2506.20869v1)
Proposed Method
Retrieval Layer
The retriever will use both sparse keyword matching and dense semantic embedding search, then merge results using a hybrid fusion strategy such as reciprocal-rank-style ranking or reranking. This design helps preserve exact term matching while also capturing conceptually similar documents. [superlinked](https://superlinked.com/vectorhub/articles/optimizing-rag-with-hybrid-search-reranking)
Semantic Reward System
A scoring module will measure retrieval usefulness using relevance labels or weak supervision, then assign positive reward to related content and negative reward to irrelevant content. This can be implemented with similarity thresholds, pairwise ranking loss, or reinforcement-style feedback so the retriever learns to prefer evidence that actually supports the query. [arxiv](https://arxiv.org/html/2507.22716v1)
Compression Layer
After retrieval, a ReComp-inspired compression module will shorten long evidence contexts while preserving critical facts for generation. This will help keep prompts concise, reduce latency, and improve faithful generation under long-document conditions. [neurips](https://neurips.cc/virtual/2024/poster/96497)
Generation Layer
The generator will answer only from the compressed retrieved evidence, with citation-style grounding if required by the application. This should improve transparency and reduce hallucinations compared with unguided generation. [toloka](https://toloka.ai/blog/rag-evaluation-a-technical-guide-to-measuring-retrieval-augmented-generation/)
Evaluation Plan
The system will be evaluated on two levels: retrieval quality and answer quality. Retrieval will be measured with relevance-oriented metrics such as precision, recall, MRR, and nDCG, while generation will be measured with exact match, F1, faithfulness, and human judgment for groundedness. [arxiv](https://arxiv.org/html/2504.14891v1)
A domain-wise benchmark will be created for:
- Architecture.
- Law.
- Medicine.
- Finance.
- Computer-related topics. [arxiv](https://arxiv.org/html/2506.20869v1)
For each domain, results will be reported separately so the system’s strengths and weaknesses are visible across fields. The main success criterion is to reach above 90% accuracy on general context tasks, while also maintaining strong domain-specific performance. [arxiv](https://arxiv.org/html/2506.20869v1)
Expected Outcomes
The proposed system is expected to improve retrieval relevance, reduce unrelated passage selection, and produce more faithful answers under long-context settings. It should also provide a clearer performance profile across domains, which is important because RAG evaluation is highly domain-dependent. [arxiv](https://arxiv.org/html/2409.13385v1)
Title:Enhancement of a Hybrid Retrieval-Augmented Generation System with Keyword and Semantic Search, Reward-Shaped Retrieval, and ReComp-Based Context Compression. [keyvalue](https://www.keyvalue.systems/blog/hybrid-rag-architecture/)
This project proposes a hybrid RAG architecture that combines keyword-based retrieval with semantic search, adds a reward-and-punishment scheme to improve retrieval relevance, and uses ReComp-style context compression to handle long contexts more efficiently. [arxiv](https://arxiv.org/html/2409.13385v1)
Background
Retrieval-Augmented Generation is widely used to reduce LLM limitations such as knowledge cut-off, hallucinations, and lack of transparency, because it grounds generation in retrieved external evidence. Hybrid search is especially useful because keyword retrieval improves exact-match precision while semantic retrieval helps recover conceptually related content. [toloka](https://toloka.ai/blog/rag-evaluation-a-technical-guide-to-measuring-retrieval-augmented-generation/)
Context compression methods such as ReComp-like approaches are important because they reduce prompt size while trying to preserve faithful information from retrieved documents. Recent work also shows that retrieval systems can benefit from reward-based reasoning and reflection mechanisms that explicitly encourage sufficient and accurate evidence use. [arxiv](https://arxiv.org/html/2507.22716v1)
Problem Statement
Current RAG systems often retrieve partially relevant or unrelated passages, and this harms answer quality even when the generation model is strong. The challenge is not only to increase recall, but also to score retrieved items so that related content is rewarded and unrelated content is penalized, especially in the semantic retrieval stage. [superlinked](https://superlinked.com/vectorhub/articles/optimizing-rag-with-hybrid-search-reranking)
Objectives
- Build a hybrid retriever that combines keyword search and semantic search into one ranking pipeline. [keyvalue](https://www.keyvalue.systems/blog/hybrid-rag-architecture/)
- Design a semantic relevance scoring mechanism that rewards closely related passages and penalizes unrelated passages. [arxiv](https://arxiv.org/html/2507.22716v1)
- Add ReComp-based long-context compression so that retrieved evidence can be condensed without losing faithfulness. [iclr](https://iclr.cc/virtual/2024/poster/17885)
- Evaluate the system with a target of above 90% accuracy on general context tasks. [arxiv](https://arxiv.org/html/2504.14891v1)
- Test performance separately across architecture, law, medicine, finance, and computer-related domains, and record the results by domain. [arxiv](https://arxiv.org/html/2506.20869v1)
Proposed Method
Retrieval Layer
The retriever will use both sparse keyword matching and dense semantic embedding search, then merge results using a hybrid fusion strategy such as reciprocal-rank-style ranking or reranking. This design helps preserve exact term matching while also capturing conceptually similar documents. [superlinked](https://superlinked.com/vectorhub/articles/optimizing-rag-with-hybrid-search-reranking)
Semantic Reward System
A scoring module will measure retrieval usefulness using relevance labels or weak supervision, then assign positive reward to related content and negative reward to irrelevant content. This can be implemented with similarity thresholds, pairwise ranking loss, or reinforcement-style feedback so the retriever learns to prefer evidence that actually supports the query. [arxiv](https://arxiv.org/html/2507.22716v1)
Compression Layer
After retrieval, a ReComp-inspired compression module will shorten long evidence contexts while preserving critical facts for generation. This will help keep prompts concise, reduce latency, and improve faithful generation under long-document conditions. [neurips](https://neurips.cc/virtual/2024/poster/96497)
Generation Layer
The generator will answer only from the compressed retrieved evidence, with citation-style grounding if required by the application. This should improve transparency and reduce hallucinations compared with unguided generation. [toloka](https://toloka.ai/blog/rag-evaluation-a-technical-guide-to-measuring-retrieval-augmented-generation/)
Evaluation Plan
The system will be evaluated on two levels: retrieval quality and answer quality. Retrieval will be measured with relevance-oriented metrics such as precision, recall, MRR, and nDCG, while generation will be measured with exact match, F1, faithfulness, and human judgment for groundedness. [arxiv](https://arxiv.org/html/2504.14891v1)
A domain-wise benchmark will be created for:
- Architecture.
- Law.
- Medicine.
- Finance.
- Computer-related topics. [arxiv](https://arxiv.org/html/2506.20869v1)
For each domain, results will be reported separately so the system’s strengths and weaknesses are visible across fields. The main success criterion is to reach above 90% accuracy on general context tasks, while also maintaining strong domain-specific performance. [arxiv](https://arxiv.org/html/2506.20869v1)
Expected Outcomes
The proposed system is expected to improve retrieval relevance, reduce unrelated passage selection, and produce more faithful answers under long-context settings. It should also provide a clearer performance profile across domains, which is important because RAG evaluation is highly domain-dependent. [arxiv](https://arxiv.org/html/2409.13385v1)