AffectFlow-DINO

AffectFlow-DINO

Uncertainty-aware multi-task facial affect estimation for the 11th ABAW Multi-Task Learning challenge: DINOv3 backbone + deterministic VA/expression/AU heads + a conditional rectified-flow head over the joint 22-dimensional affect vector.

Code, training scripts, and the full 28-experiment ablation study: github.com/Bekhouche/AffectFlow-DINO.

Paper: AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow — Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Fadi Dornaika, Abdenour Hadid. arXiv:2607.13250, 2026.

Quickstart

pip install torch torchvision transformers huggingface_hub pillow
git clone https://github.com/Bekhouche/AffectFlow-DINO.git && cd AffectFlow-DINO
python inference.py --model finetune-flow-retune-b10 --image face.jpg

Models in this repo

Each subfolder is one checkpoint: model.pt (weights + architecture config) plus, where applicable, au_thresholds.json and/or expr_weights.json post-hoc calibration files that inference.py applies automatically.

P_MTL = P_VA + P_EXPR + P_AU is the official ABAW MTL composite metric (P_VA = mean CCC of valence/arousal, P_EXPR = expression macro-F1, P_AU = mean AU F1), evaluated on the s-Aff-Wild2 validation split. Official challenge baseline: P_MTL = 0.450.

These 8 checkpoints are a curated subset of the ~30 ablation runs behind the paper — one per architecturally distinct configuration. The full ablation table, including sweeps that reused one of these architectures with different loss weights/regularization, is in the GitHub repo's EXPERIMENTS.md.

Input / output

• Input: a cropped, roughly frontal face image, resized to 224x224 and ImageNet-normalized (handled automatically by inference.py).
• Output: valence, arousal in [-1, 1]; one of 8 expressions (Neutral, Anger, Disgust, Fear, Happiness, Sadness, Surprise, Other); a subset of 12 active Action Units (AU1, AU2, AU4, AU6, AU7, AU10, AU12, AU15, AU23, AU24, AU25, AU26).
• Two decode modes: deterministic (task heads directly) or flow (average of N sampled rectified-flow trajectories — enables uncertainty estimation via trajectory spread).

Limitations

Trained and validated only on s-Aff-Wild2 (in-the-wild but frame-level, no temporal context). Expression prediction remains the weakest task (P_EXPR <= 0.35): Fear and Sadness are rare classes that only partially recover under calibration. Not evaluated on demographic subgroups; treat outputs as research signals, not clinical or high-stakes decisions.

Citation

@article{bekhouche2026affectflowdino,
  title         = {AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow},
  author        = {Bekhouche, Salah Eddine and Sellam, Abdellah Zakaria and Dornaika, Fadi and Hadid, Abdenour},
  journal       = {arXiv preprint arXiv:2607.13250},
  year          = {2026}
}

License

MIT.