Multimodal Diabetic Retinopathy Detection

I am putting together a small educational AI/ML project that detects diabetic retinopathy on the publicly-available OLIVES dataset. The goal is not only to build a working multimodal model (fundus images plus any supporting clinical metadata you find useful) but also to showcase clear explainability and rigorous evaluation so the project can be presented in an academic setting.

Scope of work
• Prepare the OLIVES dataset, handle any class imbalance, and document the preprocessing pipeline.
• Design and train a multimodal architecture of your choice in Python—PyTorch, TensorFlow or another modern framework is fine—as long as the code is clean and reproducible.
• Produce the quantitative metrics I need: Accuracy, Precision, F1-score, AUC and Cohen Kappa on a held-out test set.
• Provide the qualitative/explainability outputs I will showcase: Confusion matrix, ROC curve, PR curves with calibration plots, Dice score, IoU, feature-importance analysis, attention visualisations and Grad-CAM heat-maps overlaid on the retinal images.
• Summarise everything in a concise report or notebook: data splits, model architecture, training details, results discussion and key takeaways.

Acceptance criteria
1. Jupyter notebook (or script + README) that runs end-to-end on a fresh environment.
2. Saved model weights and inference script that reproduces the metrics above.
3. Folder of visual outputs (plots and heat-maps) clearly labelled.
4. Short written explanation (≈2 pages) of how each explainability tool supports clinical interpretability.

This is intentionally a compact project, perfect for educational presentation, so clear code organisation and well-commented rationale matter just as much as raw performance.