Image Data Analysis
Budget: ₹1,500 – ₹12,500 INR
Project Title: Medical Image Captioning with Explainability and Optimization
Overview: We are seeking a skilled AI/ML freelancer with strong experience in encoder-decoder models, medical image captioning, and model optimization techniques. The goal is to fine-tune and extend an existing architecture to generate accurate and explainable captions for medical images.
Key Requirements:
Dataset: Use the ROCO dataset (80,000+ medical images) for training and evaluation.
Model: Fine-tune the MedICap model integrating:
SCST (Self-Critical Sequence Training)
Cross Entropy Loss
BERTScore
Optimization: Implement Hybrid Harris Hawk Optimization (HHO) for feature selection to improve model efficiency.
Explainability: Integrate Grad-CAM++ to generate visual explanations (heatmaps) and compare performance with and without explainability.
Evaluation Metrics:
Captioning: BERTScore, BLEU-4, ROUGE-L, METEOR, CIDEr
Efficiency: Accuracy, Feature Selection Time, Computational Time
Implementation Scope:
End-to-end model training and fine-tuning
Image upload interface
Caption generation and Grad-CAM++ visualization integration
medicap -> https://github.com/aehrc/imageclefmedical_caption_23
Deliverables:
Fully trained and fine-tuned model
Optimized and explainable pipeline
Clean, modular code with documentation
? Technical Skills
Deep Learning & Computer Vision
Proficiency with encoder-decoder architectures (e.g., CNN-RNN, Transformer-based models)
Experience with image captioning tasks
Understanding of SCST (Self-Critical Sequence Training)
Natural Language Processing
Familiarity with BERTScore, BLEU, ROUGE, METEOR, CIDEr
Handling of text generation tasks and NLP metrics
Optimization Algorithms
Knowledge of metaheuristic optimization techniques, especially Harris Hawk Optimization (HHO)
Experience applying optimization to feature selection
Model Explainability
Expertise with Grad-CAM++ or similar explainability tools for vision models
Frameworks & Libraries
PyTorch / Lightning
Transformers (Hugging Face)
OpenCV, Matplotlib (for visualizations)
scikit-learn, NumPy, Pandas
DevOps & Integration
Python scripting and modular code structuring
Experience creating web or script-based interfaces (e.g., Streamlit, Flask, or basic UI)
Model deployment & integration (e.g., upload image → output caption + Grad-CAM++)
Overview: We are seeking a skilled AI/ML freelancer with strong experience in encoder-decoder models, medical image captioning, and model optimization techniques. The goal is to fine-tune and extend an existing architecture to generate accurate and explainable captions for medical images.
Key Requirements:
Dataset: Use the ROCO dataset (80,000+ medical images) for training and evaluation.
Model: Fine-tune the MedICap model integrating:
SCST (Self-Critical Sequence Training)
Cross Entropy Loss
BERTScore
Optimization: Implement Hybrid Harris Hawk Optimization (HHO) for feature selection to improve model efficiency.
Explainability: Integrate Grad-CAM++ to generate visual explanations (heatmaps) and compare performance with and without explainability.
Evaluation Metrics:
Captioning: BERTScore, BLEU-4, ROUGE-L, METEOR, CIDEr
Efficiency: Accuracy, Feature Selection Time, Computational Time
Implementation Scope:
End-to-end model training and fine-tuning
Image upload interface
Caption generation and Grad-CAM++ visualization integration
medicap -> https://github.com/aehrc/imageclefmedical_caption_23
Deliverables:
Fully trained and fine-tuned model
Optimized and explainable pipeline
Clean, modular code with documentation
? Technical Skills
Deep Learning & Computer Vision
Proficiency with encoder-decoder architectures (e.g., CNN-RNN, Transformer-based models)
Experience with image captioning tasks
Understanding of SCST (Self-Critical Sequence Training)
Natural Language Processing
Familiarity with BERTScore, BLEU, ROUGE, METEOR, CIDEr
Handling of text generation tasks and NLP metrics
Optimization Algorithms
Knowledge of metaheuristic optimization techniques, especially Harris Hawk Optimization (HHO)
Experience applying optimization to feature selection
Model Explainability
Expertise with Grad-CAM++ or similar explainability tools for vision models
Frameworks & Libraries
PyTorch / Lightning
Transformers (Hugging Face)
OpenCV, Matplotlib (for visualizations)
scikit-learn, NumPy, Pandas
DevOps & Integration
Python scripting and modular code structuring
Experience creating web or script-based interfaces (e.g., Streamlit, Flask, or basic UI)
Model deployment & integration (e.g., upload image → output caption + Grad-CAM++)