Train & Evaluate Chinese AVSR

I have a mouth-cropped MP4 dataset with Mandarin transcripts already cleaned and aligned. What I need now is a solid training-and-evaluation pass on three architectures—auto-avsr, AV-HuBERT and an existing AVSR baseline—so we can understand their true accuracy and, most importantly, their character error rate (CER).

Because the video data are fully pre-processed, you can head straight into fine-tuning. Accuracy and a low CER are the only performance metrics that matter for this round; inference speed can be dealt with later. I will provide the dataset via cloud storage along with any scripts I have used so far. You’re free to keep, improve, or completely replace those scripts as long as everything remains reproducible in PyTorch (Lightning welcome) and CUDA-ready.

Deliverables
• Training code or notebooks that run end-to-end on a clean environment
• Checkpoints for each model after training
• A concise evaluation report comparing CER and overall accuracy on the test split, plus a brief note on the hyper-parameters and design choices you made
• (Optional) short recommendations on the quickest wins for further accuracy gains

Acceptance criteria: the code reproduces the reported CER within ±0.2 % on my machine, and all three models are evaluated under identical conditions.