RAG + VLLM Batch Processing Data Retrieval System App update
Budget: $250 – $750 USD
Hello, we have a rag based Data Retrieval app. Our goal is to enhance performance & reduce latency to near zero. I will provide my current front-end and back-end.
Notes:
-We will not change llm models.
-We won't use any external apis.
-We will not touch our rag system (we have graph rag + vactor rag).
-We can't do anything that will sacrifice/reduce accuracy of our search or retrieval.
Recommendation of work:
- Optimize database queries and file handling with asynchronous libraries.
- Reduce LLM request timeouts and cache responses.
- Restrict CORS to trusted domains.
- Optimize Uvicorn settings for better concurrency.
- Profile and benchmark the application to identify bottlenecks.
Notes:
-We will not change llm models.
-We won't use any external apis.
-We will not touch our rag system (we have graph rag + vactor rag).
-We can't do anything that will sacrifice/reduce accuracy of our search or retrieval.
Recommendation of work:
- Optimize database queries and file handling with asynchronous libraries.
- Reduce LLM request timeouts and cache responses.
- Restrict CORS to trusted domains.
- Optimize Uvicorn settings for better concurrency.
- Profile and benchmark the application to identify bottlenecks.
Related categories:
Machine Learning (ML)
Deep Learning
LLM Prompt Engineering
Retrieval-Augmented Generation (RAG)