Real-Time AI Phase 2
Budget: $750 – $1,500 USD
I’m ready to move into Phase 2 of my conversational AI roadmap and need a developer who can turn real-time replies and long-term memory into a production-ready feature set.
The goal
• Replies must be generated and delivered in under a second so the conversation feels truly live.
• The system should remember prior exchanges (across sessions, not just within the current chat) and surface that context automatically to improve future responses.
Current state
Phase 1 delivered core LLM integration and basic chat functionality. The codebase is cleanly modular (Python backend with a lightweight React front end) and already running on a small Kubernetes cluster. You’ll receive full repo access plus API keys for OpenAI and any supporting services currently in use.
What I need from you
1. Real-time streaming endpoint (WebSocket or Server-Sent Events) that pushes incremental token updates to the client.
2. Memory layer implemented via a scalable store of your choice (e.g., Redis, Supabase, PostgreSQL with pgvector, Pinecone). It must:
• write each user-AI turn with minimal latency
• retrieve relevant historical context efficiently (embedding similarity or keyed look-ups)
3. Context-aware prompt assembly pipeline that blends new input with retrieved memory before calling the LLM.
4. Security and privacy controls: rate-limiting, token auth, and per-user data isolation.
5. Clear documentation and a one-click deployment script (Helm chart update or Docker Compose) so I can reproduce the environment.
Success criteria
• End-to-end latency ≤ 1000 ms for typical prompts (measured in my staging environment).
• Memory retrieval accuracy demonstrably improves response relevance in A/B tests.
• Codebase passes my existing unit tests plus any new ones you add.
• A concise README and architecture diagram make future phases straightforward.
If you’ve built anything similar—especially streaming chat or vector-based memory—please share links or repos. Looking forward to working together on this next milestone.
The goal
• Replies must be generated and delivered in under a second so the conversation feels truly live.
• The system should remember prior exchanges (across sessions, not just within the current chat) and surface that context automatically to improve future responses.
Current state
Phase 1 delivered core LLM integration and basic chat functionality. The codebase is cleanly modular (Python backend with a lightweight React front end) and already running on a small Kubernetes cluster. You’ll receive full repo access plus API keys for OpenAI and any supporting services currently in use.
What I need from you
1. Real-time streaming endpoint (WebSocket or Server-Sent Events) that pushes incremental token updates to the client.
2. Memory layer implemented via a scalable store of your choice (e.g., Redis, Supabase, PostgreSQL with pgvector, Pinecone). It must:
• write each user-AI turn with minimal latency
• retrieve relevant historical context efficiently (embedding similarity or keyed look-ups)
3. Context-aware prompt assembly pipeline that blends new input with retrieved memory before calling the LLM.
4. Security and privacy controls: rate-limiting, token auth, and per-user data isolation.
5. Clear documentation and a one-click deployment script (Helm chart update or Docker Compose) so I can reproduce the environment.
Success criteria
• End-to-end latency ≤ 1000 ms for typical prompts (measured in my staging environment).
• Memory retrieval accuracy demonstrably improves response relevance in A/B tests.
• Codebase passes my existing unit tests plus any new ones you add.
• A concise README and architecture diagram make future phases straightforward.
If you’ve built anything similar—especially streaming chat or vector-based memory—please share links or repos. Looking forward to working together on this next milestone.
Related categories:
JavaScript
Python
Software Architecture
Redis
Git
Kubernetes
AI Chatbot Development
AI Development