Amphora_NeuroText

Amphora NeuroText v4 — Audio & Text → Brain ROI Activation

Small MLP brain encoder: predict which brain regions activate in response to any audio or text — no fMRI required at inference time.

Trained exclusively on real naturalistic fMRI data.

This is a demo, not representative of the full model**.

Predicts activation across 56 canonical brain ROIs (HCP MMP1.0 parcellation).

---

Headline Result

Audio model (Whisper v4) beats TRIBE v2 — the Meta AI model that won the Algonauts 2025 competition — by +4.2% on a 23-subject cross-subject holdout:

All 7 brain networks exceeded TRIBE v2 at a fraction of the compute. Biggest leads: Frontal (+0.209), Default Mode (+0.221), Subcortical (+0.234). These predictions are made on a coarser 56 ROI level, future work will be devoted towards increasing prediction fidelity.

---

Models

Val R = mean Pearson R across 56 ROIs on held-out subjects (honest — see below). Holdout R = independent eval on 23 never-seen subjects.

Note on previous models (v2/v3): Models in this repo before July 2026 reported inflated val_R values (0.239–0.413) due to within-subject train/val splits. Those numbers are not comparable to v4. The v4 models use honest per-subject z-scoring and per-subject holdout splits.

---

Why These Numbers Are Honest

Previous NeuroText versions had inflated val_R from a within-subject split: the model saw the same subjects in both train and val, and learned subject-level baseline BOLD activations. Cross-subject, that memorized baseline is useless.

v4 fixes both issues:

1. Per-subject z-scoring — each subject's ROI activations are z-scored independently before training, so the model learns stimulus-driven fluctuations rather than who each subject is.
2. Per-subject holdout split — 15% of subjects per dataset are excluded from training entirely. Zero subject overlap guaranteed.
3. Single shared model — no per-subject adaptation at inference. TRIBE requires fine-tuning on each test subject; Amphora does not.

---

Audio Model — Top ROIs (held-out eval)

53/56 ROIs R>0.10 · 41/56 R>0.20 · 20/56 R>0.30

---

Quick Start

from huggingface_hub import hf_hub_download
from predict import predict_audio, predict_text, top_rois

REPO = "ffh92r32rm0/Amphora_NeuroText"

# Audio → brain regions (recommended)
ckpt = hf_hub_download(REPO, "text2roi_whisper_v4.pt")
roi_map = predict_audio("clip.wav", ckpt)
print(top_rois(roi_map, n=5))
# → [('ACC', 0.44), ('STG', 0.42), ('Thalamus', 0.41), ...]

# Text → brain regions
ckpt = hf_hub_download(REPO, "text2roi_combined_v4.pt")
roi_map = predict_text("I am terrified of the dark", ckpt)
print(top_rois(roi_map, n=5))
# → [('Amygdala_L', 0.xx), ('AI', 0.xx), ('dACC', 0.xx), ...]

CLI

# Audio
python predict.py audio clip.wav --model text2roi_whisper_v4.pt --top 10

# Text
python predict.py text "watching a spider crawl toward me" --model text2roi_combined_v4.pt

# Text + Audio
python predict.py combined "narration text" clip.wav

---

Training Details

• Corpus: 2.73M TRs · 289 GB · 4,480 sessions (CNeuroMod Friends, Narratives, LPP, HCP, language fMRI, Cowen-Keltner)
• Brain space: fsaverage5 + subcortical, 28,444 vertices → 56 ROI parcellation (HCP MMP1.0)
• Architecture: Linear(in_dim→1024) → GELU → Dropout → LayerNorm → Linear(1024→512) → GELU → Dropout → Linear(512→56)
• Loss: Pearson R (+ anchor ranking for qwen3 model)
• Epochs: 120 per model
• Eval metric: Mean Pearson R across 56 ROIs, identical to Algonauts 2025

---

Requirements

torch>=2.0
transformers>=4.40
huggingface_hub>=0.23
numpy>=1.24
librosa>=0.10   # for audio loading

---

Citation / Contact

Built by Amphora.

If you use this in research, please cite the HuggingFace repo URL.