AI Live Video Mood Editor
Budget: $30 – $250 USD
I’m building a mobile app that lets users press “record”, speak for 30 – 60 seconds, and watch the clip transform in real time. While the camera is rolling, the app must
• analyse their voice on-device, determine the mood or tone, and
• swap the live background with a matching “text cartoon” animation,
• layer mixed-reality effects (think floating emojis, comic-style captions, subtle 3D particles) that react to the words being spoken.
When the recording stops, the final clip is instantly shareable and auto-deletes after a preset countdown. The entire editing pipeline has to feel instantaneous; no post-processing screens, no waiting bars.
Core objectives
1. Real-time voice analysis → mood classification (happy, calm, excited, etc.).
2. Dynamic chroma-free background replacement with the chosen text-cartoon animations.
3. Mixed-reality overlays that update frame-by-frame according to detected keywords or sentiment.
4. Lightweight architecture so videos render smoothly on modern iOS and Android devices.
5. Temporary storage logic: clips live on the server/local cache only for the configured lifespan before permanent deletion.
Preferred tech (feel free to suggest improvements): on-device ML models via TensorFlow Lite or Core ML for sentiment; OpenCV or MediaPipe for foreground segmentation; ARKit/ARCore or Unity for mixed-reality layers; standard social-share intents for output.
Deliverables
• Fully functional prototype (iOS and/or Android) with the above pipeline.
• Source code and build instructions.
• README explaining how to retrain the sentiment model and add new background packs.
• Brief test report demonstrating live editing performance on at least one mid-range device.
If you have proven experience with real-time video processing, AR/mixed reality, or on-device AI, I’d love to see what you can put together.
• analyse their voice on-device, determine the mood or tone, and
• swap the live background with a matching “text cartoon” animation,
• layer mixed-reality effects (think floating emojis, comic-style captions, subtle 3D particles) that react to the words being spoken.
When the recording stops, the final clip is instantly shareable and auto-deletes after a preset countdown. The entire editing pipeline has to feel instantaneous; no post-processing screens, no waiting bars.
Core objectives
1. Real-time voice analysis → mood classification (happy, calm, excited, etc.).
2. Dynamic chroma-free background replacement with the chosen text-cartoon animations.
3. Mixed-reality overlays that update frame-by-frame according to detected keywords or sentiment.
4. Lightweight architecture so videos render smoothly on modern iOS and Android devices.
5. Temporary storage logic: clips live on the server/local cache only for the configured lifespan before permanent deletion.
Preferred tech (feel free to suggest improvements): on-device ML models via TensorFlow Lite or Core ML for sentiment; OpenCV or MediaPipe for foreground segmentation; ARKit/ARCore or Unity for mixed-reality layers; standard social-share intents for output.
Deliverables
• Fully functional prototype (iOS and/or Android) with the above pipeline.
• Source code and build instructions.
• README explaining how to retrain the sentiment model and add new background packs.
• Brief test report demonstrating live editing performance on at least one mid-range device.
If you have proven experience with real-time video processing, AR/mixed reality, or on-device AI, I’d love to see what you can put together.