Audio Sync React Native Client Development
Budget: $250 – $750 USD
React Native Audio Mapping Data Generation Client Development
1. Project Overview
This project aims to develop a client-side feature using React Native Bare Workflow that simultaneously captures the phone’s microphone input (user speech combined with speaker loopback) and speaker output (primarily TTS synthesized speech or other audio/music played on the phone). The two audio streams will be synchronized, and mapping data will be generated. The generated mapping data will then be transmitted to the server via the existing websocket communication module (data transmission implementation is not required).
2. Key Requirements
2.1 Functional Requirements
Audio Capture
Microphone Audio Capture:
Capture the phone's microphone input (recording) data in real time.
The microphone input will include the user’s speech as well as the speaker output captured in loopback format.
Speaker Audio Capture:
Capture the phone’s speaker output (playback) data in real time.
The speaker output primarily consists of TTS synthesized audio or other sounds/music played on the phone.
Captured Audio Format:
Sample Rate: 16,000 Hz
Bit Depth: Signed 16bits
Channels: Mono
Format: Linear PCM, raw audio format
Frame-Based Processing:
Audio data must be processed in frames of 2048 samples (2048 samples = 4096 bytes).
Both microphone and speaker audio should be picked up simultaneously in frames of 2048 samples for synchronization processing.
Synchronization and Timestamping
Assign accurate timestamps to each frame of the two audio streams (microphone and speaker) to achieve synchronization.
Utilize synchronization algorithms (e.g., buffering, timestamp sorting, etc.) to generate mapping data between the two signals.
The generated mapping data must include each audio frame's timestamp, sample rate, channel information, and the relative time difference between the microphone and speaker streams.
Mapping Data Generation
Ultimately, the synchronized data should be generated in JSON or a specified binary format.
The generated mapping data will be transmitted to the server using the existing websocket communication module.
2.2 Technical Requirements
Platform: React Native Bare Workflow (Expo not used)
Native Module Development:
iOS: Swift or Objective-C
Android: Java or Kotlin
Develop a native module for capturing speaker audio and integrate it via the React Native Bridge.
Audio Processing and Synchronization:
Implement timestamping and buffering-based synchronization logic on the client side, processing the captured audio in frames (2048 samples).
The implementation should combine JavaScript and native code.
Data Format:
Mapping data should be generated in JSON or a specified binary format, including timestamp, sample rate, and channel information.
Version Control: Use Git with GitHub or GitLab for collaboration.
3. Candidate Eligibility and Urgency
Eligibility:
Only developers who have previous experience with similar projects (audio capture, synchronization, and native module development in React Native) should apply.
Timeline:
The project timeline is extremely tight and must be completed within 3 days. Only those who are confident in delivering a working solution within this timeframe should apply.
4. Expected Deliverables
Complete source code and project files (including the React Native project and native modules for both iOS and Android).
Native module source code for speaker audio capture (iOS and Android).
Documentation for the audio synchronization and mapping data generation algorithms.
Installation and build guide documentation.
API specifications and test result reports.
1. Project Overview
This project aims to develop a client-side feature using React Native Bare Workflow that simultaneously captures the phone’s microphone input (user speech combined with speaker loopback) and speaker output (primarily TTS synthesized speech or other audio/music played on the phone). The two audio streams will be synchronized, and mapping data will be generated. The generated mapping data will then be transmitted to the server via the existing websocket communication module (data transmission implementation is not required).
2. Key Requirements
2.1 Functional Requirements
Audio Capture
Microphone Audio Capture:
Capture the phone's microphone input (recording) data in real time.
The microphone input will include the user’s speech as well as the speaker output captured in loopback format.
Speaker Audio Capture:
Capture the phone’s speaker output (playback) data in real time.
The speaker output primarily consists of TTS synthesized audio or other sounds/music played on the phone.
Captured Audio Format:
Sample Rate: 16,000 Hz
Bit Depth: Signed 16bits
Channels: Mono
Format: Linear PCM, raw audio format
Frame-Based Processing:
Audio data must be processed in frames of 2048 samples (2048 samples = 4096 bytes).
Both microphone and speaker audio should be picked up simultaneously in frames of 2048 samples for synchronization processing.
Synchronization and Timestamping
Assign accurate timestamps to each frame of the two audio streams (microphone and speaker) to achieve synchronization.
Utilize synchronization algorithms (e.g., buffering, timestamp sorting, etc.) to generate mapping data between the two signals.
The generated mapping data must include each audio frame's timestamp, sample rate, channel information, and the relative time difference between the microphone and speaker streams.
Mapping Data Generation
Ultimately, the synchronized data should be generated in JSON or a specified binary format.
The generated mapping data will be transmitted to the server using the existing websocket communication module.
2.2 Technical Requirements
Platform: React Native Bare Workflow (Expo not used)
Native Module Development:
iOS: Swift or Objective-C
Android: Java or Kotlin
Develop a native module for capturing speaker audio and integrate it via the React Native Bridge.
Audio Processing and Synchronization:
Implement timestamping and buffering-based synchronization logic on the client side, processing the captured audio in frames (2048 samples).
The implementation should combine JavaScript and native code.
Data Format:
Mapping data should be generated in JSON or a specified binary format, including timestamp, sample rate, and channel information.
Version Control: Use Git with GitHub or GitLab for collaboration.
3. Candidate Eligibility and Urgency
Eligibility:
Only developers who have previous experience with similar projects (audio capture, synchronization, and native module development in React Native) should apply.
Timeline:
The project timeline is extremely tight and must be completed within 3 days. Only those who are confident in delivering a working solution within this timeframe should apply.
4. Expected Deliverables
Complete source code and project files (including the React Native project and native modules for both iOS and Android).
Native module source code for speaker audio capture (iOS and Android).
Documentation for the audio synchronization and mapping data generation algorithms.
Installation and build guide documentation.
API specifications and test result reports.