Implement the code for Zero-Shot Text-Based Insertion described in a research paper in Python / Tensorflow.
Budget: $250 – $750 USD
The paper "Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration" (https://arxiv.org/abs/2109.05426) describes a way to do text based editing on an audio file. You take a sentence "I like basketball" and can edit it by inserting an additional word "I like PLAYING basketball". Please have a look at the paper to understand the complexity of the project before replying. There is some basic torch code available (see link above) and a youtube summary where you can listen to some outputs: https://www.youtube.com/watch?v=BVlAizncrFA
What I require from this project:
- Implement the paper using python and Tensorflow (no torch)
- Be able to run this code as easily as possible using a single method that has as few inputs as possible. For instance (if that's possible): a path to the audio file, the transcript of the audio file, the transcript that includes the new word to be added. The output should be a .wav file or the MEL spectrogram used to syntethise it
- Be able to do all necessary training locally - Have a simple setup and instructions that allows me to run local training where necessary
- Be able to run it locally, not on google colab
- A quick assessment on what would be required to have it run in a different language
When evaluating I will:
- Look whether the paper was implemented as described
- Check whether I can reproduce the results
What I require from this project:
- Implement the paper using python and Tensorflow (no torch)
- Be able to run this code as easily as possible using a single method that has as few inputs as possible. For instance (if that's possible): a path to the audio file, the transcript of the audio file, the transcript that includes the new word to be added. The output should be a .wav file or the MEL spectrogram used to syntethise it
- Be able to do all necessary training locally - Have a simple setup and instructions that allows me to run local training where necessary
- Be able to run it locally, not on google colab
- A quick assessment on what would be required to have it run in a different language
When evaluating I will:
- Look whether the paper was implemented as described
- Check whether I can reproduce the results