EEG-MRI Transformer for Psychiatric Diagnosis

I need a visual-transformer–based pipeline that can take raw EEG recordings and structural MRI scans, fuse the two modalities, and output a clear diagnostic decision for psychiatric disorders. The end goal is a medical-grade tool that helps clinicians distinguish between healthy controls and patients, so accuracy and interpretability matter more to me than sheer benchmark speed.

Here is what I already have and what I expect from you:

• Well-curated datasets of synchronized EEG and MRI are available under research licences; I will provide access links once the project starts.
• You will design or adapt a Vision Transformer (ViT) backbone, convert the EEG into image-like representations (e.g., time–frequency spectrograms or topographic maps) and merge them with MRI slices or 3-D volumes using an attention-based fusion strategy.
• The model should train end-to-end in PyTorch (preferred) with clear scripts for preprocessing, training, validation and inference.
• Performance must be reported with standard clinical metrics—AUC, sensitivity, specificity—on a held-out test set.
• I need concise documentation so that hospital staff can reproduce the results, plus a short technical report explaining the architecture choices and how the attention maps can be visualised for clinical insight.

If you have prior experience with medical imaging, EEG feature engineering, or multimodal transformers, I’d like to see examples. Otherwise, let me know how you plan to tackle regulatory-grade data handling and the small-sample challenges inherent to psychiatric datasets.

Deliverables that will mark the job complete:
1. Full, commented source code and environment file.
2. Trained model weights and a reproducible inference notebook.
3. Documentation and the technical report described above.

Looking forward to collaborating on a model that could genuinely improve psychiatric diagnosis.