iOS Engineer: Real-Time Camera Processing & Hybrid AI Engine (Swift + AVFoundation)
Budget: $30 – $250 USD
Project Overview:
We are looking for an experienced iOS Engineer to build the core architecture of a Real-Time Computer Vision Application.
The app requires a high-performance video pipeline that captures frames, runs local inference (CoreML), and selectively sends data to a Cloud LLM API.
This is a pure engineering challenge. We need clean, modular code that handles concurrency, thermal management, and hardware switching without draining the battery instantly.
Phase 1 Scope (MVP):
You will build a functional "Skeleton App" focusing on performance and stability using the internal camera.
Key Technical Requirements:
Advanced Camera & Audio Pipeline (AVFoundation):
Implement a robust AVCaptureSession to handle video buffers (CMSampleBuffer).
CRITICAL: The architecture must support Hot-Plugging External Devices.
Requirement: When an external USB device is connected, the app must automatically switch BOTH the Video Feed AND the Audio Input to the external source.
Use Case: The device (e.g., smart glasses or webcam) will act as the primary sensor for both vision and sound.
Note: Use standard AVCaptureDevice.DiscoverySession (type .external).
Hybrid AI Logic (The Core Engine):
Local: Run a lightweight CoreML model for real-time object detection (offline).
Cloud: Integrate a REST/WebSocket API (Gemini/GPT-4o) to analyze frames every few seconds.
Smart Switching: Use NWPathMonitor to fallback to "Local-Only" mode if the connection drops.
Thermal & Performance Optimization:
Implement Adaptive Throttling: Monitor ProcessInfo.processInfo.thermalState.
Logic: If the device gets hot (.serious state), automatically reduce the Cloud API request rate while keeping the local model active.
Background Execution:
Ensure the app’s audio engine (TTS) remains active even when the screen is locked or the app is in the background.
Required Tech Stack:
Swift 5+, UIKit or SwiftUI.
AVFoundation (Deep understanding of capture sessions, inputs, and audio routing).
CoreML & Vision Framework.
Concurrency (GCD / Swift Actors).
Deliverables:
A clean Xcode project (GitHub repo).
The "Hybrid Engine" working smoothly on a physical device.
Code documentation explaining the modular camera/audio setup.
We are looking for an experienced iOS Engineer to build the core architecture of a Real-Time Computer Vision Application.
The app requires a high-performance video pipeline that captures frames, runs local inference (CoreML), and selectively sends data to a Cloud LLM API.
This is a pure engineering challenge. We need clean, modular code that handles concurrency, thermal management, and hardware switching without draining the battery instantly.
Phase 1 Scope (MVP):
You will build a functional "Skeleton App" focusing on performance and stability using the internal camera.
Key Technical Requirements:
Advanced Camera & Audio Pipeline (AVFoundation):
Implement a robust AVCaptureSession to handle video buffers (CMSampleBuffer).
CRITICAL: The architecture must support Hot-Plugging External Devices.
Requirement: When an external USB device is connected, the app must automatically switch BOTH the Video Feed AND the Audio Input to the external source.
Use Case: The device (e.g., smart glasses or webcam) will act as the primary sensor for both vision and sound.
Note: Use standard AVCaptureDevice.DiscoverySession (type .external).
Hybrid AI Logic (The Core Engine):
Local: Run a lightweight CoreML model for real-time object detection (offline).
Cloud: Integrate a REST/WebSocket API (Gemini/GPT-4o) to analyze frames every few seconds.
Smart Switching: Use NWPathMonitor to fallback to "Local-Only" mode if the connection drops.
Thermal & Performance Optimization:
Implement Adaptive Throttling: Monitor ProcessInfo.processInfo.thermalState.
Logic: If the device gets hot (.serious state), automatically reduce the Cloud API request rate while keeping the local model active.
Background Execution:
Ensure the app’s audio engine (TTS) remains active even when the screen is locked or the app is in the background.
Required Tech Stack:
Swift 5+, UIKit or SwiftUI.
AVFoundation (Deep understanding of capture sessions, inputs, and audio routing).
CoreML & Vision Framework.
Concurrency (GCD / Swift Actors).
Deliverables:
A clean Xcode project (GitHub repo).
The "Hybrid Engine" working smoothly on a physical device.
Code documentation explaining the modular camera/audio setup.