Python Coder for MMT(Multimodal Machine Translation) Implementation
Budget: ₹600 – ₹1,500 INR
I need you to implement a multimodal machine translation(Using image to improve the translation quality) task using the Bridgetower vision language model, accessible at https://huggingface.co/docs/transformers/model_doc/bridgetower and https://github.com/microsoft/BridgeTower?tab=readme-ov-file. Your task involves taking a paragraph containing multiple sentences and its corresponding image as input. Fuse these inputs using the Calixto Multimodal NMT model, found at https://github.com/iacercalixto/MultimodalNMT. Additionally, for English to Hindi translation, leverage the MBart model at https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt. Ensure that the entire code is executed within a Google Colab environment, with a primary focus on fine-tuning the Bridgetower model for Multimodal Machine Translation (MMT) tasks.
Related categories:
Python
Machine Learning (ML)
Natural Language
Computer Vision
Multimodal Large Language Model