Build AI Avatar for Pharma Training Platform
Budget: $3,000 – $5,000 USD
Project Title:
Build a Real-Time AI Avatar Role-Play Platform (LiveKit + HeyGen + OpenAI RAG)
Project Description:
The Goal: We are building SkillDose, a premium training platform for pharmaceutical sales representatives. We need a Senior Full-Stack AI Engineer to develop a high-fidelity MVP where users can practice sales pitches via voice with an interactive, hyper-realistic AI Avatar acting as a doctor.
The Core Challenge: The interaction must be real-time (<800ms latency) and feel natural. The avatar must listen, "think" based on a closed knowledge base (PDFs), and respond with synchronized lip-sync video.
Required Architecture (Non-Negotiable): We are looking for a developer with proven experience implementing the LiveKit Agents Framework.
Orchestrator: LiveKit Agents (Python or Node.js) to handle the real-time pipeline.
Visual Avatar: Integration with HeyGen Interactive Avatar SDK (via LiveKit plugin).
Intelligence (Brain): OpenAI GPT-4o connected to a Vector Database (Pinecone/Milvus) for RAG (Retrieval-Augmented Generation). The avatar must only use provided medical documents to answer.
Voice Pipeline: Deepgram (STT) and ElevenLabs (TTS) optimized for low latency.
Key Features to Build:
Real-Time Voice Conversation: Users speak naturally; the avatar responds instantly.
Barge-In Capability: If the user interrupts the avatar, the avatar must stop speaking immediately (handled via LiveKit).
Semantic Evaluation Logic: A background agent must analyze the user's speech and score it against a "Correct Answer" rubric (not keyword matching, but semantic meaning).
Persona Implementation: Configure one specific avatar persona ("Dr. Skeptic") with a strict personality and objection-handling logic defined in our prompt system.
Deliverables:
A functional Web App (React/Next.js) with the Avatar interface.
Backend infrastructure set up (LiveKit Server + Agents).
Integration of 1 Medical Product PDF (Knowledge Base).
Source code documentation and deployment guide.
Ideal Candidate:
You have built Voice AI Agents before using LiveKit.
You understand WebSockets and WebRTC nuances.
You have experience with RAG pipelines (LangChain/LlamaIndex) to prevent AI hallucinations.
Bonus: Previous experience with HeyGen or Synthesia APIs.
Skills Required (Selecciona estas etiquetas):
Artificial Intelligence (AI)
Python (or Node.js)
OpenAI API
React.js
VoIP / WebRTC
Build a Real-Time AI Avatar Role-Play Platform (LiveKit + HeyGen + OpenAI RAG)
Project Description:
The Goal: We are building SkillDose, a premium training platform for pharmaceutical sales representatives. We need a Senior Full-Stack AI Engineer to develop a high-fidelity MVP where users can practice sales pitches via voice with an interactive, hyper-realistic AI Avatar acting as a doctor.
The Core Challenge: The interaction must be real-time (<800ms latency) and feel natural. The avatar must listen, "think" based on a closed knowledge base (PDFs), and respond with synchronized lip-sync video.
Required Architecture (Non-Negotiable): We are looking for a developer with proven experience implementing the LiveKit Agents Framework.
Orchestrator: LiveKit Agents (Python or Node.js) to handle the real-time pipeline.
Visual Avatar: Integration with HeyGen Interactive Avatar SDK (via LiveKit plugin).
Intelligence (Brain): OpenAI GPT-4o connected to a Vector Database (Pinecone/Milvus) for RAG (Retrieval-Augmented Generation). The avatar must only use provided medical documents to answer.
Voice Pipeline: Deepgram (STT) and ElevenLabs (TTS) optimized for low latency.
Key Features to Build:
Real-Time Voice Conversation: Users speak naturally; the avatar responds instantly.
Barge-In Capability: If the user interrupts the avatar, the avatar must stop speaking immediately (handled via LiveKit).
Semantic Evaluation Logic: A background agent must analyze the user's speech and score it against a "Correct Answer" rubric (not keyword matching, but semantic meaning).
Persona Implementation: Configure one specific avatar persona ("Dr. Skeptic") with a strict personality and objection-handling logic defined in our prompt system.
Deliverables:
A functional Web App (React/Next.js) with the Avatar interface.
Backend infrastructure set up (LiveKit Server + Agents).
Integration of 1 Medical Product PDF (Knowledge Base).
Source code documentation and deployment guide.
Ideal Candidate:
You have built Voice AI Agents before using LiveKit.
You understand WebSockets and WebRTC nuances.
You have experience with RAG pipelines (LangChain/LlamaIndex) to prevent AI hallucinations.
Bonus: Previous experience with HeyGen or Synthesia APIs.
Skills Required (Selecciona estas etiquetas):
Artificial Intelligence (AI)
Python (or Node.js)
OpenAI API
React.js
VoIP / WebRTC
Related categories:
Python
Node.js
React.js
Software Documentation
Deep Learning
ElevenLabs
AI Model Integration