LLM RAG Vector Database Setup
Budget: $30 – $250 USD
I’m building a retrieval-augmented generation (RAG) pipeline and need a specialist to stand up the vector database layer for my large-language-model workflow. All content going into the store will be purely textual—think markdown files, knowledge-base articles, and long-form documents—so the schema, chunking strategy, and embedding approach should be optimised for fast, accurate text search.
Here’s what I’d like from you:
• Recommend and deploy a production-ready vector database (Pinecone, Weaviate, Chroma, Milvus or a comparable option).
• Design a text-specific embedding and metadata schema, including parameters such as chunk size, overlap, and namespace strategy.
• Build ingestion scripts that batch-process my existing documents, generate embeddings (OpenAI, Hugging Face or similar), and populate the database.
• Provide a lightweight retrieval module that I can call from my LLM layer to perform similarity search, filter on metadata, and return ranked contexts.
• Supply concise documentation so I can extend the pipeline or change providers later.
Clean, well-commented Python preferred, but I’m open to Node.js if that fits your toolchain better. Please highlight any prior RAG or vector search projects you’ve delivered and note the database(s) you’re most comfortable with.
Here’s what I’d like from you:
• Recommend and deploy a production-ready vector database (Pinecone, Weaviate, Chroma, Milvus or a comparable option).
• Design a text-specific embedding and metadata schema, including parameters such as chunk size, overlap, and namespace strategy.
• Build ingestion scripts that batch-process my existing documents, generate embeddings (OpenAI, Hugging Face or similar), and populate the database.
• Provide a lightweight retrieval module that I can call from my LLM layer to perform similarity search, filter on metadata, and return ranked contexts.
• Supply concise documentation so I can extend the pipeline or change providers later.
Clean, well-commented Python preferred, but I’m open to Node.js if that fits your toolchain better. Please highlight any prior RAG or vector search projects you’ve delivered and note the database(s) you’re most comfortable with.