Standalone LiveKit Voice AI Agent
Budget: ₹1,500 – ₹12,500 INR
I am building a fully self-hosted, real-time voice assistant on top of LiveKit. The system must ingest the caller’s audio stream, perform on-premise voice recognition, run natural-language processing with contextual comprehension, and respond intelligently as a first-line customer-support agent—all without relying on any third-party SaaS such as Vapi, ElevenLabs, or similar hosted APIs. Open-source or locally runnable models are fine; cloud subscriptions are not.
Core flow
• LiveKit delivers the audio track in real time.
• Your pipeline converts speech to text, keeps multi-turn context, and routes it through an NLP layer capable of understanding intent and maintaining conversation state.
• The agent then generates the textual reply that my existing interface will present. (I do not need text-to-speech for this phase.)
What I will review as deliverables
1. Docker-compose (or similar) scripts that spin up the LiveKit server plus your speech-to-text and NLP services.
2. Source code with clear README showing how audio frames are captured from LiveKit, transcribed, and processed for contextual responses.
3. A sample conversation demonstrating successful handling of at least three consecutive customer-support queries where context is preserved.
4. Brief performance metrics: average latency from spoken word to textual reply under typical load.
Acceptance criteria
• Runs on my Ubuntu 22.04 box with a single “docker compose up”.
• No outbound calls to commercial speech or language APIs during operation.
• Accuracy and context retention good enough to answer common support questions (I will provide a small FAQ to test).
Mention any preferred open-source models or frameworks when you reply so I can ensure licensing compatibility with my product.
Core flow
• LiveKit delivers the audio track in real time.
• Your pipeline converts speech to text, keeps multi-turn context, and routes it through an NLP layer capable of understanding intent and maintaining conversation state.
• The agent then generates the textual reply that my existing interface will present. (I do not need text-to-speech for this phase.)
What I will review as deliverables
1. Docker-compose (or similar) scripts that spin up the LiveKit server plus your speech-to-text and NLP services.
2. Source code with clear README showing how audio frames are captured from LiveKit, transcribed, and processed for contextual responses.
3. A sample conversation demonstrating successful handling of at least three consecutive customer-support queries where context is preserved.
4. Brief performance metrics: average latency from spoken word to textual reply under typical load.
Acceptance criteria
• Runs on my Ubuntu 22.04 box with a single “docker compose up”.
• No outbound calls to commercial speech or language APIs during operation.
• Accuracy and context retention good enough to answer common support questions (I will provide a small FAQ to test).
Mention any preferred open-source models or frameworks when you reply so I can ensure licensing compatibility with my product.