AI Avatar with Real-Time Voice Interaction

I am looking for a freelance developer or team to create a local AI avatar system with real-time voice interaction and facial/lip synchronization.

Currently, we already have a basic avatar that can display responses, but it does not speak or animate facial movements naturally.

The goal is to build an avatar that can:

Speak directly using AI-generated voice (TTS)
Synchronize mouth/facial movements with speech
Simulate realistic modulation using at least the 5 main vowel mouth shapes (visemes/phonemes)
Run locally (offline or local server environment)
Allow flexible integration with different AI providers

Main requirements:

• Local execution
The system must run locally using CPU/GPU resources. Cloud dependence should be minimal or optional.

• Lip sync / facial animation
The avatar should animate while speaking, including:

mouth movement synchronization
basic facial animation
blinking / idle movements preferred

Possible technologies are open to proposal:

Unity
Unreal Engine
Three.js
WebGL
Live2D
NVIDIA Audio2Face
Oculus LipSync
Rhubarb Lip Sync
or similar alternatives

• AI integration flexibility
The conversational AI provider is not fixed.
The architecture should allow easy replacement/integration of APIs such as:

Grok
OpenAI
Claude
Gemini
local LLMs
custom APIs

We will later modify the backend/API ourselves, so modular architecture is important.

• Audio pipeline
Ideally the system should support:

microphone input
speech-to-text
AI response generation
text-to-speech
synchronized avatar playback

Deliverables:

Fully functional prototype
Source code
Basic installation documentation
Modular architecture
Local deployment instructions

Preferred experience:

AI avatars
lip sync systems
facial animation
TTS/STT
Unity/Unreal
real-time rendering
local AI systems

Optional future features:

multiple avatars
emotions
streaming integration
facial recognition
body animation
camera integration

Please include:

technologies you would use
estimated timeline
previous related work/demo if available
approximate budget estimate for MVP development.