Mobile AI Agent with Multimodal Inputs
Budget: £250 – £750 GBP
I’m building a mobile-first AI agent that can fluidly switch between voice commands, standard text input, and basic gesture controls. The core logic, NLP pipeline, and gesture-recognition layer all need to sit inside a single, maintainable codebase that compiles cleanly for iOS and Android.
You’ll start by designing the interaction flow: how spoken intent, typed text, or a swipe/pinch maps into the same intent engine. From there, I want the full implementation—speech-to-text, intent classification, gesture mapping, and the reply generation module—wired together behind a unified API so the mobile front end can call one endpoint regardless of modality.
I’m comfortable with TensorFlow Lite or PyTorch Mobile for the on-device models and open to using platform-native voice libraries as long as latency stays low. Clean, well-commented code and concise setup documentation are essential; the finished agent should run offline for core tasks and fall back to cloud services only when absolutely necessary.
Deliverables
• Complete source code with build scripts
• Model training notebooks and exported .tflite/.pt files
• A brief README explaining app integration steps
I’ll consider the project complete once the demo app recognizes at least 90 % of test commands across all three input types and returns appropriate responses within two seconds.
You’ll start by designing the interaction flow: how spoken intent, typed text, or a swipe/pinch maps into the same intent engine. From there, I want the full implementation—speech-to-text, intent classification, gesture mapping, and the reply generation module—wired together behind a unified API so the mobile front end can call one endpoint regardless of modality.
I’m comfortable with TensorFlow Lite or PyTorch Mobile for the on-device models and open to using platform-native voice libraries as long as latency stays low. Clean, well-commented code and concise setup documentation are essential; the finished agent should run offline for core tasks and fall back to cloud services only when absolutely necessary.
Deliverables
• Complete source code with build scripts
• Model training notebooks and exported .tflite/.pt files
• A brief README explaining app integration steps
I’ll consider the project complete once the demo app recognizes at least 90 % of test commands across all three input types and returns appropriate responses within two seconds.