Senior Voice AI & Omnichannel Automation Engineer (LiveKit + Vobiz + Sarvam + WhatsApp & Email) -- 2
Budget: ₹1,500 – ₹12,500 INR
We are developing an omnichannel automated outreach and inbound conversational SaaS platform using React, Express.js, and Node.js. Our platform seamlessly connects a real-time conversational voice bot ecosystem with bulk WhatsApp and email marketing engines.
Our SaaS platform frontend is already completely designed and built. The backend real-time architecture is functional, but we need an experienced engineer to make surgical improvements to our existing functionalities. You will optimize the system for ultra-low latency, implement flawless voice interruption mechanics using LiveKit, Vobiz SIP, and Sarvam AI, and deploy our bulk messaging features.
Technical Challenges & Scope of Work
1. Voice Bot Optimization (LiveKit + Vobiz + Sarvam AI) [1]
• Stream & Pipeline Piping: Optimize the existing voice pipeline using the LiveKit Agents framework. Convert our voice flow (using Vobiz SIP, Sarvam STT/TTS, and Groq LLM) into an end-to-end chunk-streaming architecture to eliminate the current 4–7 second delay. [1, 2, 3, 4]
• Barge-In & Interruption Handling: Resolve the broken "pipeline busy → buffering utterance" logic. Use LiveKit's turn-taking engine and Voice Activity Detection (VAD) to build a cancel-and-restart flow. It must instantaneously flush audio buffers and kill in-flight API generations the millisecond a user speaks. [1]
• Memory Management: Fix backend EventEmitter memory leak warnings, clean up stale WebSocket/RTP event listeners, and completely prevent old utterances from replaying out of sync.
2. Bulk WhatsApp Business API Integration
• Campaign Engine: Modify existing functionality to support contact database uploads, triggering automated bulk campaigns, and sending outbound messages via the WhatsApp Business API using Virtual IP numbers.
• Flow Triggers: Connect incoming WhatsApp user replies to automatically trigger outbound conversational AI voice calls or programmatic follow-ups based on keywords.
• WhatsApp Analytics Dashboard: Link performance metrics to our frontend, showing data on total messages sent, delivered, opened, and active conversation reply rates.
3. Bulk Email & AI Agent Conversational Flow [1]
• Database Blasts: Support contact database uploads for massive bulk email outreach campaigns.
• Inbound AI Conversations: Wire the conversational AI backend to scan incoming email responses and automatically continue the conversation over text threads.
• Draft Approval Mode: Update the existing dashboard controls to add a global toggle switch. When enabled, incoming AI-generated email responses are saved straight to the user's email drafts folder for manual human review before sending.
• Email Analytics Dashboard: Fully connect backend event metrics to our frontend dashboard to monitor real-time email delivery states, open rates, click-through rates, and active AI thread statuses.
Technical Stack & Requirements
• Core Frameworks: LiveKit SDK & Plugins (livekit-agents, @livekit/agents-plugin-sarvam), Express.js, and Node.js streams.
• Telephony & Media: Vobiz SIP trunks, WebSockets, RTP packet streaming, and low-latency audio buffer manipulation.
• AI Ecosystem: Groq API, Sarvam AI Bulbul (TTS) & Saaras (STT) models.
• Omnichannel APIs: WhatsApp Business Cloud API, Meta developer ecosystem, SMTP, and IMAP protocols. [1, 2, 3]
Expected Deliverables
• Near real-time conversational voice latency with immediate barge-in response using LiveKit.
• Backend modifications that connect to our pre-built frontend UI dashboards for both WhatsApp and Email metrics.
• Working draft approval toggle infrastructure for automated email responses.
• Production-ready refactored backend code and clean implementation documentation. [1, 2, 3]
When Applying Please Include
1. Deployed examples of real-time voice bots utilizing LiveKit along with telephony integrations.
2. Your technical approach to implementing stream buffer flushes on customer barge-in inside Node.js.
3. Your experience linking bulk campaign metrics from WhatsApp Business and Email protocols to an existing React frontend.
4. Your estimated timeline and budget for updating these backend functionalities. [1, 2, 3]
Proactive Next Steps
To speed up onboarding once you choose a freelancer, would you like me to:
• Outline the exact LiveKit agent configuration object needed to stream Sarvam TTS and STT simultaneously?
• Structure the API schema required to push real-time email dashboard metrics directly to your existing React frontend?
• Write a conceptual Node.js middleware script showing how the email draft approval toggle intercepts the LLM output?
Our SaaS platform frontend is already completely designed and built. The backend real-time architecture is functional, but we need an experienced engineer to make surgical improvements to our existing functionalities. You will optimize the system for ultra-low latency, implement flawless voice interruption mechanics using LiveKit, Vobiz SIP, and Sarvam AI, and deploy our bulk messaging features.
Technical Challenges & Scope of Work
1. Voice Bot Optimization (LiveKit + Vobiz + Sarvam AI) [1]
• Stream & Pipeline Piping: Optimize the existing voice pipeline using the LiveKit Agents framework. Convert our voice flow (using Vobiz SIP, Sarvam STT/TTS, and Groq LLM) into an end-to-end chunk-streaming architecture to eliminate the current 4–7 second delay. [1, 2, 3, 4]
• Barge-In & Interruption Handling: Resolve the broken "pipeline busy → buffering utterance" logic. Use LiveKit's turn-taking engine and Voice Activity Detection (VAD) to build a cancel-and-restart flow. It must instantaneously flush audio buffers and kill in-flight API generations the millisecond a user speaks. [1]
• Memory Management: Fix backend EventEmitter memory leak warnings, clean up stale WebSocket/RTP event listeners, and completely prevent old utterances from replaying out of sync.
2. Bulk WhatsApp Business API Integration
• Campaign Engine: Modify existing functionality to support contact database uploads, triggering automated bulk campaigns, and sending outbound messages via the WhatsApp Business API using Virtual IP numbers.
• Flow Triggers: Connect incoming WhatsApp user replies to automatically trigger outbound conversational AI voice calls or programmatic follow-ups based on keywords.
• WhatsApp Analytics Dashboard: Link performance metrics to our frontend, showing data on total messages sent, delivered, opened, and active conversation reply rates.
3. Bulk Email & AI Agent Conversational Flow [1]
• Database Blasts: Support contact database uploads for massive bulk email outreach campaigns.
• Inbound AI Conversations: Wire the conversational AI backend to scan incoming email responses and automatically continue the conversation over text threads.
• Draft Approval Mode: Update the existing dashboard controls to add a global toggle switch. When enabled, incoming AI-generated email responses are saved straight to the user's email drafts folder for manual human review before sending.
• Email Analytics Dashboard: Fully connect backend event metrics to our frontend dashboard to monitor real-time email delivery states, open rates, click-through rates, and active AI thread statuses.
Technical Stack & Requirements
• Core Frameworks: LiveKit SDK & Plugins (livekit-agents, @livekit/agents-plugin-sarvam), Express.js, and Node.js streams.
• Telephony & Media: Vobiz SIP trunks, WebSockets, RTP packet streaming, and low-latency audio buffer manipulation.
• AI Ecosystem: Groq API, Sarvam AI Bulbul (TTS) & Saaras (STT) models.
• Omnichannel APIs: WhatsApp Business Cloud API, Meta developer ecosystem, SMTP, and IMAP protocols. [1, 2, 3]
Expected Deliverables
• Near real-time conversational voice latency with immediate barge-in response using LiveKit.
• Backend modifications that connect to our pre-built frontend UI dashboards for both WhatsApp and Email metrics.
• Working draft approval toggle infrastructure for automated email responses.
• Production-ready refactored backend code and clean implementation documentation. [1, 2, 3]
When Applying Please Include
1. Deployed examples of real-time voice bots utilizing LiveKit along with telephony integrations.
2. Your technical approach to implementing stream buffer flushes on customer barge-in inside Node.js.
3. Your experience linking bulk campaign metrics from WhatsApp Business and Email protocols to an existing React frontend.
4. Your estimated timeline and budget for updating these backend functionalities. [1, 2, 3]
Proactive Next Steps
To speed up onboarding once you choose a freelancer, would you like me to:
• Outline the exact LiveKit agent configuration object needed to stream Sarvam TTS and STT simultaneously?
• Structure the API schema required to push real-time email dashboard metrics directly to your existing React frontend?
• Write a conceptual Node.js middleware script showing how the email draft approval toggle intercepts the LLM output?