VLbai-2.6AD

VLbai-2.6AD

A clinical reasoning assistant for early-stage Alzheimer's assessment. It joins a 3D MRI + biomarker classifier (Vbai-2.6AD) to a reasoning LLM (Gemma 4 12B) inside a single forward pass — the diagnosis is passed as vectors, not text.

Answers are in Turkish.

Research use only. Not a medical device. Not validated for clinical decision-making. See Limitations.

---

What This Is (and is not)

Most "LLM + medical model" systems are pipelines: the classifier prints a label, the label is pasted into a prompt, the LLM writes prose around it. The LLM never sees the evidence — only a word.

Here, Vbai-2.6AD's fused representation (1280-d) is projected into 4 soft tokens that live in Gemma's embedding space and are marked as non-text via mm_token_type_ids=1. Gemma attends to the representation, not to a string.

  3D MRI (T1 96³)                13 Biomark + Masks
       │                                   │
ResNet3D + CBAM/SE/ASPP             TabularEncoder
       │                                   │
       └───────        CrossModalFusion      ───────┐
                                │                  │
                       Combined Vector (1280-d)  Classification Header
                                │                (CN / MCI / AD & 5-Year Risk)
                       Projector (LN→Dense...)     │
                                │                  │
                       4 Soft Token (3840-d)       │
                                │                  │
[Text Embedding] ++ [SOFT TOKENS] ++ [Text Embedding] ──► Gemma 12B + LoRA
                                                                │
                                                     Turkish Clinical Description

The classifier's verdict is an anchor: the LLM explains and defends it, but cannot overturn it. Numbers come from code, prose comes from the LLM.

Evidence That the Vector Channel is Real

Zeroing the soft tokens while keeping the text prompt identical changes the stated class in 47.5% of cases (19/40). If this were a pipeline dressed up as fusion, the text alone would carry the answer and ablation would change nothing.

---

Results

Evaluated on a held-out test split (40 unique patients, eval_faithfulness.py).

Classification and Early Risk Model General Tests (Only Vbai-2.6AD)

Ablation of the classifier itself, before any LLM involvement:

MRI changes 20/123 predictions relative to biomarkers alone (14 better, 6 worse).

*Tested with ADNI T1 and ADNIMERGE values datasets. But training is excluding ADNI T1 and ADNIMERGE values datasets.

*It was trained in just 14 epochs.

*No transfer learning or pre-trained weights were used.

Catastrophic Forgetting

LoRA adapts the model to a narrow task. These benchmarks measure how much general biomedical reasoning was lost — they are not a measure of this model's clinical ability.

Equal magnitude in opposite directions, both inside the ±0.029 standard error at n=300. No measurable forgetting. We attribute this to the multitask training data (8 task types) rather than to a single templated objective.

Run-to-Run Variance

LoRA training here is not stable across seeds. On a later data revision we trained three seeds and observed:

Validation loss barely moved while behaviour swung by 20 points. The released checkpoint is a single run; treat its numbers as one draw from this distribution, not as a stable expectation. --seed is exposed in train_projector.py for reproduction.

---

Usage

Load Gemma yourself, then apply the adapter and projector. Merged weights are deliberately not distributed (see License).

python chat.py \
  --features vlbai-2.6ad_features_tbm.pt \
  --text    vlbai-2.6ad_text.json \
  --projector projector.pt

Two modes:

• report — thinking off, short answers, questions chosen from a menu of 8 canonical questions (/questions). Free-typed questions are allowed but warned about.
• discuss — thinking on, long-form reasoning, free input.

Why the menu exists

LoRA learns phrase patterns, not task boundaries. A free-typed question that falls between trained patterns gets answered with a blend of them, which occasionally produced self-contradictory text ("atrophy is marked" and "all values normal" in one answer). Restricting rapor mode to in-distribution questions removes the failure mechanism rather than patching its symptoms.

---

Training data

817 ADNI patients → 7353 supervised examples across 8 task types:

Every target is rule-derived from the data — none are LLM-synthesized. This is deliberate: a synthesized target teaches the model to sound confident about numbers it did not compute. Digits appear in targets only where the code owns them (probs, ROI z-scores).

The absent task exists because of a real failure: when a clinician said "there is a family history of early-onset Alzheimer's", an earlier model replied that "the model assessment includes these findings". It did not. Accepting a false premise and reasoning on top of it is the most dangerous error class in a clinical tool.

Region measurements come from ADNI's UCSFFSX7 FreeSurfer table, ICV-normalized, region codes resolved by name from DATADIC.rda. 811/817 patients matched with a median scan-to-table interval of 0 days.

Generated datasets are not redistributed — they are ADNI-derived. The build scripts are included so holders of ADNI access can regenerate them.

---

Limitations

• Single cohort. 817 ADNI patients. No external validation. Performance on other scanners, populations, or preprocessing pipelines is unknown.
• Modality is not interchangeable. This checkpoint was trained on TBM Jacobian volumes. Feeding raw T1 produces confident nonsense. The extraction scripts require an explicit --tbm / --t1 flag for this reason.
• The absent task can undermine its own anchor. In this checkpoint, the closing sentence of absent targets ends on the word "assessment", and the model has been observed to blend it with the hold task into "the Vbai-2.6AD assessment can be changed if clinically meaningful" — which contradicts the anchor rule. rapor mode's menu prevents this in practice; a caller bypassing the menu is not protected. Fixed in the data generator after this checkpoint was trained.
• One patient reproducibly breaks. 005_S_0324 produced corrupted output (stray non-Turkish tokens, leaked control text) in 4/4 training runs. Suspected outlier feature vector. Not yet diagnosed.
• Single training run. See the variance table above.
• Turkish only. Prompts and targets are Turkish; other languages are untested.
• Not a medical device. No regulatory clearance. Research use only.

---

Evaluation scripts

There is no accepted benchmark for "MRI-grounded Alzheimer's assistant". These scripts fill that gap and are included so results can be reproduced or disputed:

---

License

Two layers, both apply:

1. Projector, training code, evaluation scripts, data generators — original work, released under CC-BY-NC-4.0.
2. LoRA adapter — a derivative of Gemma 4. Google's Gemma Terms of Use govern it and are not overridden by the license above. You must comply with both.

Base model weights are not redistributed. Download Gemma from Google under its own terms.

ADNI. These weights encode information derived from ADNI data. Use is subject to the ADNI Data Use Agreement. Redistribution of ADNI-derived datasets is not permitted here; generated training files are excluded from this repository for that reason.

Acknowledgements

Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI). ADNI investigators contributed to the design and implementation of ADNI and provided data but did not participate in the analysis or writing of this work.

Requirements

• Python ≥ 3.9
• PyTorch ≥ 2.0
• CUDA-capable GPU, ≥ 16 GB VRAM recommended (Tested with at least an NVIDIA L4 GPU with 24 GB of VRAM) (Trained with NVIDIA A100 with of 40 GB of VRAM)
• See requirements.txt for full dependency list

Support

• Website: Neurazum
• Email: [email protected]

---