Offline MetaHuman Conversational Tutor
Budget: $250 – $750 USD
I want to speak into a microphone, have a fully local stack turn my words into a MetaHuman that answers back in real-time, all on Windows. The speech-to-text (Whisper.cpp), text-to-speech (Piper) and viseme-driven lip sync parts are relatively straightforward; what I need most is a rock-solid, entirely offline chat brain. I’m leaning toward llama.cpp with a 7-8 B instruct model, but I’m open to your suggestions on model choice, quantisation and prompt engineering as long as everything stays on-device.
Here’s the flow I expect you to wire together, optimise and document:
• Microphone → Whisper.cpp
• Transcript → llama.cpp (local LLM)
• Response → Piper TTS
• Audio → Unreal Engine playback + MetaHuman lip sync (method still open—OVRLipSync style visemes or NVIDIA Audio2Face if you can keep latency low)
Key expectations
• Runs on a single Windows machine without internet access
• Latency low enough for a conversational feel inside UE
• Clear setup scripts or blueprints I can reproduce on a clean system
• Advice on model weights, GPU/CPU balance and any trade-offs you make
When you reply, focus on your hands-on experience with local LLM deployment, Unreal Engine audio pipelines, and any prior work syncing MetaHuman faces to speech. If you’ve already built something similar, that’s exactly what I need to see.
Here’s the flow I expect you to wire together, optimise and document:
• Microphone → Whisper.cpp
• Transcript → llama.cpp (local LLM)
• Response → Piper TTS
• Audio → Unreal Engine playback + MetaHuman lip sync (method still open—OVRLipSync style visemes or NVIDIA Audio2Face if you can keep latency low)
Key expectations
• Runs on a single Windows machine without internet access
• Latency low enough for a conversational feel inside UE
• Clear setup scripts or blueprints I can reproduce on a clean system
• Advice on model weights, GPU/CPU balance and any trade-offs you make
When you reply, focus on your hands-on experience with local LLM deployment, Unreal Engine audio pipelines, and any prior work syncing MetaHuman faces to speech. If you’ve already built something similar, that’s exactly what I need to see.