Micro-Expression Recognition Dissertation

I am completing a dissertation on facial micro-expression recognition using the CASME-II dataset and need a turnkey research package. The scientific focus is emotion classification, and I want the study driven primarily by machine-learning techniques. Concretely, I’d like to explore both a conventional pipeline (hand-crafted features + classic classifiers) and modern representation learning with CNN and Transformer architectures so we can compare performance and draw strong conclusions.

Here is what I expect:

• Python code (OpenCV, scikit-learn, PyTorch or TensorFlow) that handles data preparation, frame-level preprocessing, sequence aggregation, feature extraction, and classification.
• A suite of experiments: subject-independent k-fold validation on CASME-II, clear metric reporting (accuracy, F1, confusion matrices), and statistical significance checks to show whether CNN or Transformer-based features truly outperform the standard baselines.
• A fully written dissertation manuscript—introduction through conclusion—including literature review, methodology, experimental results, discussion, and properly formatted references and in-text citations. Harvard or APA style is fine, so long as it is consistent.
• Reproducibility materials: commented notebooks or scripts, requirements.txt, and a concise README that lets an external examiner run every experiment end-to-end.

The code should be clean, modular, and well documented; the thesis text should flow naturally, weaving the experimental findings into the narrative. Once complete, I’ll run your scripts myself and cross-check the write-up against the generated figures and tables.

Here is detailed info on what i want:

Project Description
I am looking for an experienced Deep Learning and Computer Vision Engineer to implement and evaluate a modular spatiotemporal Micro-Expression Recognition (MER) pipeline in PyTorch.

The scope of this project is strictly focused on executing a systematic ablation study on the CASME II dataset. You will implement a preprocessing pipeline, a hybrid CNN-Transformer architecture, and run a systematic sweep of 12 distinct configurations toggling specific modules to isolate their impact on performance.

No cross-dataset or transfer learning is required—this project is entirely self-contained on CASME II.

Technical Scope & Architecture
You will build a modular spatiotemporal network containing four toggles:

Eulerian Video Magnification (EVM) (On/Off)
Shallow 3D-CNN Backbone (STSTNet-3D) (On/Off)
SimAM 3D Attention (On/Off)
SLSTT Sequence Transformer (On/Off)
Detailed Step-by-Step Requirements
Step 1: Preprocessing & Modality Extraction Pipeline
You must write a preprocessing pipeline that converts raw RGB video sequences from CASME II into dynamic, identity-free motion tensors:

Eulerian Video Magnification (EVM): Implement spatial decomposition (using a Laplacian pyramid) and temporal bandpass filtering (ideal filter, $0.4\text{ Hz}$ to $3.0\text{ Hz}$) to amplify sub-pixel facial twitches (amplification factor $\alpha = 20.0$).
Modality Extraction:
Calculate dense Farneback Optical Flow ($u$ and $v$ channels) between consecutive frames.
Derive Optical Skin Strain ($os$) from the spatial gradients of the flow fields: $$\epsilon_{xx} = \frac{\partial u}{\partial x}, \quad \epsilon_{yy} = \frac{\partial v}{\partial y}, \quad \epsilon_{xy} = 0.5 \left( \frac{\partial u}{\partial y} + \frac{\partial v}{\partial x} \right)$$ $$os = \sqrt{\epsilon_{xx}^2 + \epsilon_{yy}^2 + \epsilon_{xy}^2}$$
Temporal Interpolation: Standardize variable video lengths to exactly $L=33$ frames (resulting in $T=32$ motion frame pairs) using linear temporal interpolation.
Storage: Output preprocessed tensors of shape [3, 32, 224, 224] (representing flow-$u$, flow-$v$, and strain channels across 32 time steps) as .npy files to disk to prevent redundant processing during training.
Step 2: Architecture Implementation (PyTorch)
Implement the core network architecture:

STSTNet-3D Backbone: Build a three-stream shallow 3D-CNN where the $u$-flow, $v$-flow, and strain channels are routed through parallel, weight-unshared branches. Use Conv3D kernels of temporal depth 1 (kernel size [1, 3, 3]) and spatial MaxPool3D [1, 2, 2] to preserve all 32 temporal frames.
SimAM 3D Attention: Implement the parameter-free SimAM module in 3D. It must calculate an analytical energy score for each neuron based on local spatial-temporal statistics to scale feature maps element-wise.
Sequence-Level Spatiotemporal Transformer (SLSTT): Apply adaptive spatial pooling to compress features to sequence tokens of shape [B, 32, 96], add deterministic sinusoidal positional encodings, and feed the sequence through a 2-layer pre-norm Sequence Transformer to model temporal progression.
Fallbacks (for Ablation Toggles):
When 3D-CNN is Off: Fallback to a spatial raw patch average pooling layer (e.g., $4 \times 4$ grid) before flattening.
When Transformer is Off: Fallback to simple temporal global average/max pooling over the 32 frames.
Step 3: Systematic Ablation Training Sweep
Implement the script to execute training and record results for the following 12 configuration cells:

Base Configurations:
config_1_pure_base (EVM: Off, SimAM: Off, CNN3D: Off, SLSTT: Off)
config_2_temporal_only (EVM: Off, SimAM: Off, CNN3D: Off, SLSTT: On)
config_4_motion_amp_base (EVM: On, SimAM: Off, CNN3D: Off, SLSTT: Off)
config_12_permutation (EVM: On, SimAM: Off, CNN3D: Off, SLSTT: On)
Spatial & Attention Configurations: 5. config_3_spatial_only (EVM: Off, SimAM: Off, CNN3D: On, SLSTT: Off) 6. config_13_permutation (EVM: On, SimAM: Off, CNN3D: On, SLSTT: Off) 7. config_5_attention_base (EVM: Off, SimAM: On, CNN3D: On, SLSTT: Off) 8. config_16_permutation (EVM: On, SimAM: On, CNN3D: On, SLSTT: Off)
Full Spatiotemporal Configurations: 9. config_7_full_no_attention (EVM: On, SimAM: Off, CNN3D: On, SLSTT: On) 10. config_9_permutation (EVM: Off, SimAM: Off, CNN3D: On, SLSTT: On) 11. config_6_full_stage2_noevm (EVM: Off, SimAM: On, CNN3D: On, SLSTT: On) 12. config_8_proposed_unified (EVM: On, SimAM: On, CNN3D: On, SLSTT: On)
Training Parameters & Constraints:

Validation Protocol: Strict Leave-One-Subject-Out (LOSO) cross-validation across all 26 subjects (26 folds) to ensure subject independence and prevent subject identity bias.
Loss Function: Class-weighted Focal Loss ($\gamma = 2.0$) with label smoothing ($0.05$) to manage dataset class imbalance.
Optimisation: AdamW optimizer with Cosine Annealing learning rate scheduler (Peak LR: 1e-4, decay to 1e-7).
Classification Target: Train on a 3-class emotion taxonomy: Positive (Happiness), Negative (Disgust, Sadness, Fear, Anger, Repression), and Surprise.
Step 4: Results Synthesis & Visualization
Save and log results (Accuracy, Macro F1, and class-specific F1 scores) for all 12 configurations to a clean summary CSV file.
Generate three diagnostic plots:
A bar chart comparing Accuracy and Macro F1 across all key configurations.
A grouped bar chart comparing per-class F1-scores.
Normalized confusion matrices for key configurations (e.g., Spatial-Only vs. Proposed Unified).
Required Deliverables
Source Code: Well-documented Python scripts containing:
Preprocessing pipeline (EVM + Optical Flow + Optical Strain + Temporal Interpolator).
PyTorch model definitions with toggle fallback logic.
Training script supporting 26-fold LOSO cross-validation.
Orchestration script to run all 12 ablation configurations sequentially.
Results Summary: The final ablation performance CSV matrix containing metrics for all 12 runs.
Plots: Automated plotting scripts that generate the performance bar charts and confusion matrices.
Readme Guide: Simple setup guide outlining how to install dependencies, structure the dataset, and run the pipeline.
Required Skills
Proficient in Python and PyTorch
Deep understanding of Computer Vision & Signal Processing (Eulerian Video Magnification, Farneback Optical Flow, Skin Strain Gradients)
Experience with video spatiotemporal architectures (3D-CNNs, Transformers, Attention modules)
Familiarity with Leave-One-Subject-Out (LOSO) cross-validation protocols.
Experience with handling class imbalance in deep learning.