fine-tuning the XLSR-Wav2vec 2.0 pre-trained model for the Turkish language and Hungarian language
Budget: $30 – $250 USD
https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_Tune_XLSR_Wav2Vec2_on_Turkish_ASR_with_%F0%9F%A4%97_Transformers.ipynb#scrollTo=LBSYoWbi-45k
This script can be used for Turkish, but a few changes and visualizations here would be better and model output and script should be able to upload my drive.
facebook/wav2vec2-large-xlsr-53 will be pre-trained model.
• Mozilla Common Voice dataset should be used to train the models
• The models must be trained using wav2vec2 architecture https://arxiv.org/abs/2006.11477
2 pre-trained models are enough to train:
o wav2vec2-xlsr-53
(https://huggingface.co/facebook/wav2vec2-large-xlsr-53)
o wav2vec2-xls-r-300m (https://huggingface.co/facebook/wav2vec2-xls-r-300m)
3) Please pay extra attention to this subsection:
You should follow this script:
https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_Tune_XLSR_Wav2Vec2_on_Turkish_ASR_with_%F0%9F%A4%97_Transformers.ipynb
Inside this script, database installation and model trainings are given in detailed way.
Inside script, database is installed in this part:
3.1) Here instead of “common_voice” dataset you should write
“mozilla-foundation/common_voice_9_0” or other versions (7,8)
All other cleaning and pre-processing steps should be the same as in script.
3.2) And here in this script you can deifne pre-trained model that you want to fine-tune
In the above picture “facebook/wav2vec2-large-xlsr-53” pre-trained model is given.
3.3) After you finish the training, last thing you need to do is to boost the final models with n-gram language model (either 4 or 5). Here is the script for it:
https://huggingface.co/blog/wav2vec2-with-ngram
This script is intended for Swedish language. For Turkish language you can use Turkish Wikipedia dump. You can find link below:
https://dumps.wikimedia.org/trwiki/latest/trwiki-latest-pages-articles.xml.bz2
You will follow the given script, but you need to use the given Turkish data above. This is the part you need to change
Or you can generate .arpa file by using this extractor directly:
https://github.com/mpoyraz/ngram-lm-wiki
To sum up, you need to run the given colab script and boost the final models with n-gram language model.
This is all about experiments.
4) At the end, you need to write results of the trained models, compare them against each other by using charts, graphs, or tables.
The models should be evaluated on 4 metrics:
word error rate (WER)
character error rate (CER).
RTF= time needed for recognizing the full test set / total length of the full test set
memory requirement = peak GPU memory load (during test)
Additionally compare the final Turkish language models with Hungarian models (minimum 2 comparative graphs). you ’need to train the model for Hungarian. I provide already trained ones below:
https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-hungarian
https://github.com/facebookresearch/voxpopuli?fbclid=IwAR2axF_Ky-ljY38B10X4e8iOFnftZ_yuQJ6ywafmSZg5tnPGPjJYLsGnJoc
check this for getting dataset for Hungarian n-gram (and also helpful script)
This script can be used for Turkish, but a few changes and visualizations here would be better and model output and script should be able to upload my drive.
facebook/wav2vec2-large-xlsr-53 will be pre-trained model.
• Mozilla Common Voice dataset should be used to train the models
• The models must be trained using wav2vec2 architecture https://arxiv.org/abs/2006.11477
2 pre-trained models are enough to train:
o wav2vec2-xlsr-53
(https://huggingface.co/facebook/wav2vec2-large-xlsr-53)
o wav2vec2-xls-r-300m (https://huggingface.co/facebook/wav2vec2-xls-r-300m)
3) Please pay extra attention to this subsection:
You should follow this script:
https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_Tune_XLSR_Wav2Vec2_on_Turkish_ASR_with_%F0%9F%A4%97_Transformers.ipynb
Inside this script, database installation and model trainings are given in detailed way.
Inside script, database is installed in this part:
3.1) Here instead of “common_voice” dataset you should write
“mozilla-foundation/common_voice_9_0” or other versions (7,8)
All other cleaning and pre-processing steps should be the same as in script.
3.2) And here in this script you can deifne pre-trained model that you want to fine-tune
In the above picture “facebook/wav2vec2-large-xlsr-53” pre-trained model is given.
3.3) After you finish the training, last thing you need to do is to boost the final models with n-gram language model (either 4 or 5). Here is the script for it:
https://huggingface.co/blog/wav2vec2-with-ngram
This script is intended for Swedish language. For Turkish language you can use Turkish Wikipedia dump. You can find link below:
https://dumps.wikimedia.org/trwiki/latest/trwiki-latest-pages-articles.xml.bz2
You will follow the given script, but you need to use the given Turkish data above. This is the part you need to change
Or you can generate .arpa file by using this extractor directly:
https://github.com/mpoyraz/ngram-lm-wiki
To sum up, you need to run the given colab script and boost the final models with n-gram language model.
This is all about experiments.
4) At the end, you need to write results of the trained models, compare them against each other by using charts, graphs, or tables.
The models should be evaluated on 4 metrics:
word error rate (WER)
character error rate (CER).
RTF= time needed for recognizing the full test set / total length of the full test set
memory requirement = peak GPU memory load (during test)
Additionally compare the final Turkish language models with Hungarian models (minimum 2 comparative graphs). you ’need to train the model for Hungarian. I provide already trained ones below:
https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-hungarian
https://github.com/facebookresearch/voxpopuli?fbclid=IwAR2axF_Ky-ljY38B10X4e8iOFnftZ_yuQJ6ywafmSZg5tnPGPjJYLsGnJoc
check this for getting dataset for Hungarian n-gram (and also helpful script)