Unreal Engine Avatar Driven by Data Streams
Budget: $3,000 – $5,000 USD
Quick Summary of Technical Details:
We have an existing “black box” (our code) that sends four data streams in parallel. Your job is to receive these streams and use them to drive a Metahuman (or similarly realistic) avatar in Unreal Engine, ultimately packaging the result as an executable for MacOS. Specifically:
1. Gestures – Data specifying the avatar’s hand/body movements (e.g., waving, pointing, bowing, etc.).
2. Real-Time Audio Input – Audio stream (no file-based audio) for lip-sync. Preferably leveraging real-time solutions like NVIDIA’s Audio2Face Live Link (or another robust method) for natural speech animation.
3. X,Y Location – Coordinates to dictate where the avatar looks.
4. Animation Toggle (Idle On/Off) – Indicates whether the avatar should be in idle mode or actively animating (talking, gesturing, etc.).
Swappable Avatar Requirement:
• We want to ensure easy swapping of the avatar if we create or import a custom avatar in the future. The system should allow us to drop in a replacement with minimal effort.
The data format for these streams is flexible; we’re open to WebSockets or any recommended method that ensures real-time performance and is fully compatible with Unreal Engine.
(Note: Our “black box” software will run on the same machine as the Unreal app.)
Full Project Requirements
1. Unreal Engine Metahuman Character (or Custom Avatar)
• Design & Development: Create a realistic avatar using Unreal Engine’s Metahuman framework or an equally detailed approach.
• Easy Replacement: The avatar pipeline should allow us to swap in a different character model with minimal effort.
• Clarification on Realism: While the system should be capable of high-fidelity output, we will likely design our own Metahuman using official Unreal tools, so ultra-realistic character design is not the primary focus.
2. Real-Time Lip-Syncing
• Real-Time Input: Use the provided audio stream for immediate lip-sync without file-based processing.
• Preferred Technology: Potentially leverage NVIDIA’s Audio2Face Live Link or an equivalent solution to ensure accurate mouth movement.
• Accuracy: Aim for natural, accurate mouth movements matching live speech, with low latency for real-time interactivity.
3. Animations
• Idle State: An animation state for when the avatar is not speaking.
• Smooth Transitions: Seamless shifts between idle, speaking, and gesturing.
• Multiple Gestures: Incorporate two to three dozen full-body gestures (e.g., waving hello, pointing, bowing) to add realism and variety.
4. Visual Quality
• High Fidelity: Match or exceed visual quality seen on platforms like iamdave.ai, virbe.ai, or digitalhumans.com.
• Facial Expressions: Realistic micro-expressions and detailed facial rigging.
• Textures & Rendering: High-resolution textures and lifelike rendering for a polished final result.
5. Customization
• Scene & Backgrounds: Allow for a few swappable background options and lighting setups.
• User Configuration: Settings to tweak lighting, environment, and other visual elements.
6. Platform Compatibility
• MacOS Executable: The final output must run smoothly on Apple Silicon (M2 or better).
• Performance Optimization: Heavy optimization isn’t our top priority given the power of Apple Silicon, but the system should still maintain real-time interaction without sacrificing too much quality.
7. Integration Support
• Collaboration: Work with our development team to integrate the avatar into our AI assistant application.
• Documentation: Provide clear instructions for implementing the avatar, handling the data streams, and swapping out the character model.
Who We’re Looking For
• Unreal Engine Expertise: Strong background in animation, Metahumans, and real-time animation.
• Lip-Sync Know-How: Familiarity with real-time solutions like NVIDIA Audio2Face or similar.
• Animation & Rigging Skills: Deep knowledge of facial and body rigging, blending animations, and expression mapping.
• Optimization Experience: Capable of delivering high-quality visuals on MacOS Apple Silicon without performance bottlenecks.
• Good Communicator: Comfortable collaborating and iterating with our team.
To Bid on This Project
1. Portfolio Samples: Share work that demonstrates relevant experience—especially real-time lip-sync or Metahuman projects.
2. Process Overview: Outline how you’d approach building this avatar system and integrating it with our code/data streams.
3. Tech & Approach: Offer recommendations for data handling (e.g., WebSockets) and real-time animation technologies (Audio2Face, etc.).
4. Timeline & Cost: Provide an estimated schedule and budget.
We have an existing “black box” (our code) that sends four data streams in parallel. Your job is to receive these streams and use them to drive a Metahuman (or similarly realistic) avatar in Unreal Engine, ultimately packaging the result as an executable for MacOS. Specifically:
1. Gestures – Data specifying the avatar’s hand/body movements (e.g., waving, pointing, bowing, etc.).
2. Real-Time Audio Input – Audio stream (no file-based audio) for lip-sync. Preferably leveraging real-time solutions like NVIDIA’s Audio2Face Live Link (or another robust method) for natural speech animation.
3. X,Y Location – Coordinates to dictate where the avatar looks.
4. Animation Toggle (Idle On/Off) – Indicates whether the avatar should be in idle mode or actively animating (talking, gesturing, etc.).
Swappable Avatar Requirement:
• We want to ensure easy swapping of the avatar if we create or import a custom avatar in the future. The system should allow us to drop in a replacement with minimal effort.
The data format for these streams is flexible; we’re open to WebSockets or any recommended method that ensures real-time performance and is fully compatible with Unreal Engine.
(Note: Our “black box” software will run on the same machine as the Unreal app.)
Full Project Requirements
1. Unreal Engine Metahuman Character (or Custom Avatar)
• Design & Development: Create a realistic avatar using Unreal Engine’s Metahuman framework or an equally detailed approach.
• Easy Replacement: The avatar pipeline should allow us to swap in a different character model with minimal effort.
• Clarification on Realism: While the system should be capable of high-fidelity output, we will likely design our own Metahuman using official Unreal tools, so ultra-realistic character design is not the primary focus.
2. Real-Time Lip-Syncing
• Real-Time Input: Use the provided audio stream for immediate lip-sync without file-based processing.
• Preferred Technology: Potentially leverage NVIDIA’s Audio2Face Live Link or an equivalent solution to ensure accurate mouth movement.
• Accuracy: Aim for natural, accurate mouth movements matching live speech, with low latency for real-time interactivity.
3. Animations
• Idle State: An animation state for when the avatar is not speaking.
• Smooth Transitions: Seamless shifts between idle, speaking, and gesturing.
• Multiple Gestures: Incorporate two to three dozen full-body gestures (e.g., waving hello, pointing, bowing) to add realism and variety.
4. Visual Quality
• High Fidelity: Match or exceed visual quality seen on platforms like iamdave.ai, virbe.ai, or digitalhumans.com.
• Facial Expressions: Realistic micro-expressions and detailed facial rigging.
• Textures & Rendering: High-resolution textures and lifelike rendering for a polished final result.
5. Customization
• Scene & Backgrounds: Allow for a few swappable background options and lighting setups.
• User Configuration: Settings to tweak lighting, environment, and other visual elements.
6. Platform Compatibility
• MacOS Executable: The final output must run smoothly on Apple Silicon (M2 or better).
• Performance Optimization: Heavy optimization isn’t our top priority given the power of Apple Silicon, but the system should still maintain real-time interaction without sacrificing too much quality.
7. Integration Support
• Collaboration: Work with our development team to integrate the avatar into our AI assistant application.
• Documentation: Provide clear instructions for implementing the avatar, handling the data streams, and swapping out the character model.
Who We’re Looking For
• Unreal Engine Expertise: Strong background in animation, Metahumans, and real-time animation.
• Lip-Sync Know-How: Familiarity with real-time solutions like NVIDIA Audio2Face or similar.
• Animation & Rigging Skills: Deep knowledge of facial and body rigging, blending animations, and expression mapping.
• Optimization Experience: Capable of delivering high-quality visuals on MacOS Apple Silicon without performance bottlenecks.
• Good Communicator: Comfortable collaborating and iterating with our team.
To Bid on This Project
1. Portfolio Samples: Share work that demonstrates relevant experience—especially real-time lip-sync or Metahuman projects.
2. Process Overview: Outline how you’d approach building this avatar system and integrating it with our code/data streams.
3. Tech & Approach: Offer recommendations for data handling (e.g., WebSockets) and real-time animation technologies (Audio2Face, etc.).
4. Timeline & Cost: Provide an estimated schedule and budget.