macOS ARM Native CLAP Application Development
Budget: $15 – $25 USD
macOS ARM Native CLAP Application for Sound FX Fine-Tuning & Smart Naming
We are seeking an experienced **Software Developer/Machine Learning Engineer** to create a dedicated, native **macOS ARM (Apple Silicon)** desktop application using Python and PyTorch's MPS backend.
This application is crucial for professional sound designers to manage and leverage their custom sound effects libraries using cutting-edge **CLAP (Contrastive Language-Audio Pre-training)** technology.
Project Goal:
Develop a user-friendly application that provides two core functionalities:
1. **Fine-Tuning:** Recursively train/fine-tune the CLAP model on a user's local sound effects library using existing file metadata.
2. **Smart Naming:** Use the resulting CLAP model to generate accurate, semantic names for new, unlabelled sound effects.
Detailed Requirements
**Phase 1: CLAP Model Training & Database Management**
The application must provide a simple UI to handle the ingestion and training process.
* **Drag & Drop Ingestion:** Allow users to **Drag & Drop** entire folders or volumes of audio files (e.g., `.wav`, `.aiff`) into the application.
* **Recursive Metadata Extraction:** The application must recursively scan the dropped folder/volume and extract metadata to create the text-audio pairs for training. This metadata should prioritize:
* **Filename/Path:** The filename and folder structure (e.g., `SciFi/Engine/Rumble_Deep.wav` $\rightarrow$ "Sci-Fi Engine Rumble Deep")
* **Embedded Metadata:** Support reading common audio metadata fields (e.g., BWF `Description`, Soundminer, or common ID3 tags) to create richer training text captions.
* **CLAP Fine-Tuning Module:** Implement the training/fine-tuning process for the **LAION-CLAP model** (or a similar high-performance PyTorch CLAP implementation).
* The fine-tuning must use the extracted metadata as the paired text data to adapt the model to the user's specific sound effects style and vocabulary.
* A progress bar or logging window must clearly display the training status (epochs, loss).
* **Model Persistence:** Save the resulting **fine-tuned CLAP model weights** locally for use in the naming phase.
**Phase 2: Semantic Sound Effect Naming**
The application must provide a functional interface for zero-shot naming.
* **Audio Input:** Allow users to import new, unlabelled audio files.
* **CLAP Inference:** Use the **fine-tuned CLAP model** to generate an embedding for the new audio file.
* **Vocabulary Matching:** Compare the audio embedding against a custom text **Vocabulary** (pre-encoded list of descriptive phrases/filenames) and calculate the **Cosine Similarity** score for all pairs.
* **Smart Naming Output:** Display the **Top 5** best-matching text phrases from the vocabulary along with their confidence scores. The user should be able to select a name to be automatically applied to the audio file's filename or metadata.
Technical Specifications (Must-Haves)
* **Target Platform:** **macOS ARM (M-series / Apple Silicon) ONLY.**
* **Core Technology:** **Python 3.10+** (or 3.12.6 as specified in the original context).
* **Frameworks:** **PyTorch** with mandatory acceleration using the **Metal Performance Shaders (MPS)** backend for GPU utilization.
* **CLAP Implementation:** Use the **LAION-CLAP** or a similarly robust, open-source CLAP repository.
* **UI:** The final deliverable must be a standalone macOS application bundle (or a well-packaged Python app with a basic GUI/TUI).
Required Skills & Experience
* **Mandatory:** Proven expertise in **Python** for desktop application development.
* **Mandatory:** Deep practical experience with **PyTorch** and building/training deep learning models.
* **Mandatory:** Direct experience working with **Contrastive Language-Audio Pre-training (CLAP)** or related models (e.g., CLIP, AudioCLIP).
* **Critical:** Demonstrated ability to build applications optimized for **macOS ARM** hardware, specifically leveraging the **PyTorch MPS** backend.
* Familiarity with audio processing, metadata standards (BWF, ID3), and file system operations on macOS.
Deliverables
1. Complete, runnable macOS ARM application bundle.
2. Well-documented source code (Python).
3. Clear instructions for setup, running the application, and the fine-tuning process.
**Please start your bid by briefly outlining your experience with CLAP and PyTorch MPS.** We are looking for a developer who understands the core machine learning task as much as the specific platform requirement.
We are seeking an experienced **Software Developer/Machine Learning Engineer** to create a dedicated, native **macOS ARM (Apple Silicon)** desktop application using Python and PyTorch's MPS backend.
This application is crucial for professional sound designers to manage and leverage their custom sound effects libraries using cutting-edge **CLAP (Contrastive Language-Audio Pre-training)** technology.
Project Goal:
Develop a user-friendly application that provides two core functionalities:
1. **Fine-Tuning:** Recursively train/fine-tune the CLAP model on a user's local sound effects library using existing file metadata.
2. **Smart Naming:** Use the resulting CLAP model to generate accurate, semantic names for new, unlabelled sound effects.
Detailed Requirements
**Phase 1: CLAP Model Training & Database Management**
The application must provide a simple UI to handle the ingestion and training process.
* **Drag & Drop Ingestion:** Allow users to **Drag & Drop** entire folders or volumes of audio files (e.g., `.wav`, `.aiff`) into the application.
* **Recursive Metadata Extraction:** The application must recursively scan the dropped folder/volume and extract metadata to create the text-audio pairs for training. This metadata should prioritize:
* **Filename/Path:** The filename and folder structure (e.g., `SciFi/Engine/Rumble_Deep.wav` $\rightarrow$ "Sci-Fi Engine Rumble Deep")
* **Embedded Metadata:** Support reading common audio metadata fields (e.g., BWF `Description`, Soundminer, or common ID3 tags) to create richer training text captions.
* **CLAP Fine-Tuning Module:** Implement the training/fine-tuning process for the **LAION-CLAP model** (or a similar high-performance PyTorch CLAP implementation).
* The fine-tuning must use the extracted metadata as the paired text data to adapt the model to the user's specific sound effects style and vocabulary.
* A progress bar or logging window must clearly display the training status (epochs, loss).
* **Model Persistence:** Save the resulting **fine-tuned CLAP model weights** locally for use in the naming phase.
**Phase 2: Semantic Sound Effect Naming**
The application must provide a functional interface for zero-shot naming.
* **Audio Input:** Allow users to import new, unlabelled audio files.
* **CLAP Inference:** Use the **fine-tuned CLAP model** to generate an embedding for the new audio file.
* **Vocabulary Matching:** Compare the audio embedding against a custom text **Vocabulary** (pre-encoded list of descriptive phrases/filenames) and calculate the **Cosine Similarity** score for all pairs.
* **Smart Naming Output:** Display the **Top 5** best-matching text phrases from the vocabulary along with their confidence scores. The user should be able to select a name to be automatically applied to the audio file's filename or metadata.
Technical Specifications (Must-Haves)
* **Target Platform:** **macOS ARM (M-series / Apple Silicon) ONLY.**
* **Core Technology:** **Python 3.10+** (or 3.12.6 as specified in the original context).
* **Frameworks:** **PyTorch** with mandatory acceleration using the **Metal Performance Shaders (MPS)** backend for GPU utilization.
* **CLAP Implementation:** Use the **LAION-CLAP** or a similarly robust, open-source CLAP repository.
* **UI:** The final deliverable must be a standalone macOS application bundle (or a well-packaged Python app with a basic GUI/TUI).
Required Skills & Experience
* **Mandatory:** Proven expertise in **Python** for desktop application development.
* **Mandatory:** Deep practical experience with **PyTorch** and building/training deep learning models.
* **Mandatory:** Direct experience working with **Contrastive Language-Audio Pre-training (CLAP)** or related models (e.g., CLIP, AudioCLIP).
* **Critical:** Demonstrated ability to build applications optimized for **macOS ARM** hardware, specifically leveraging the **PyTorch MPS** backend.
* Familiarity with audio processing, metadata standards (BWF, ID3), and file system operations on macOS.
Deliverables
1. Complete, runnable macOS ARM application bundle.
2. Well-documented source code (Python).
3. Clear instructions for setup, running the application, and the fine-tuning process.
**Please start your bid by briefly outlining your experience with CLAP and PyTorch MPS.** We are looking for a developer who understands the core machine learning task as much as the specific platform requirement.