Dinov3d-Neuro

Dinov3D-Neuro: A 3D ViT-B/16^3 for Brain MRI trained using DINOv3-style SSL

This model uses the DINOv3 architecture and self-supervised training objective, adapted to 3D volumetric input via the nrdg/dinov3d codebase, itself a fork of facebookresearch/dinov3. Dinov3D was trained from scratch on brain MRI data and contains no weights from Meta's DINOv3 checkpoints. It is not affiliated with, endorsed by, or sponsored by Meta.

---

Model Details

Model Description

A 3D Vision Transformer encoder pretrained with a DINOv3 self-supervised objective (DINO + iBOT + KoLeo) with Gram anchoring added in later training phases, and a final high resolution adaptation phase (In Progress!). The model was trained on single-channel brain MRI volumes from the FOMO300K dataset of various contrasts (T1w, T2w, FLAIR, ...). Given a preprocessed volume, the model returns a CLS token and a grid of patch tokens (792-dim each).

• Developed by: Asa Gilmore, Dr. Ariel Rokem.
• Model type: 3D Vision Transformer (self-supervised feature encoder)
• Architecture: DINOv3 ViT, adapted to 3D - See vit3d_base in dinov3d repo
• Input: Single-channel 3D volume, resampled to 1 mm isotropic spacing and padded to a multiple of the patch size (16). Trained with 128^3 global / 48^3 local crops (base pretrain + Gram-anchoring phases), later adapted to 192^3 global / 64^3 local crops in a high-resolution phase - see Training Details.
• Output: 792-dimensional embeddings: 1 CLS token + (D/16)·(H/16)·(W/16) patch tokens (no register tokens). model.forward_features(x) returns a dict including x_norm_clstoken (B×792) and x_norm_patchtokens (B×N×792).
• License: CC BY-NC-SA 4.0 (inherited from FOMO300K - see Licensing)

Model Sources

• Repository: asagilmore/dinov3d (training/modeling code)
• Architecture reference: DINOv3 (Siméoni et al., 2025)

Relationship to DINOv3

Adapted code: This model uses code adapted from facebookresearch/dinov3, which is licensed under the DINOv3 License.

weights are original work trained from scratch on brain MRI and are licensed separately under CC BY-NC-SA 4.0.

Modifications made for 3D:

Data augmentation has also been reworked with domain specific augmentations for brain MRI, more details on data augmentation can be found in the code repo.

---

Uses

Evaluation and downstream adaptation is still in progress, check back later to see more details.

How to Get Started with the Model

This model is distributed as a raw PyTorch checkpoint plus the code in this repo, as well as sharded checkpoints at the end of each training phase.

The teacher checkpoints can be loaded following the inference instructions in the repo, and the sharded checkpoints can be used to restart training for fine-tuning experiments.

The teacher checkpoints can be found in the eval folder. We include all checkpoints captured during training, but recommend using the latest one, unless doing experiments to evaluate performance over training iterations.

The ckpt directory contains the sharded checkpoints, which we include at the end of each training phase.

The layout of the checkpoints follows from the original dinov3 outputs, so their repo can be used as a rough reference for layout.

---

Training Details

Training Data

Dataset: FOMO-MRI/FOMO300K (Cerri et al., 2026)

We filtered the fomo300k dataset to include only single channel anatomical scans. The repo contains a fomo300k.json file listing all subjects used.

Preprocessing

FOMO300K is distributed as NIfTI without co-registration or skull-stripping.

• Resampling: 1 mm^3 isotropic (Spacingd, bilinear/trilinear interpolation, border padding), applied at load time via InferenceAugmentation3d / the training data pipeline
• Padding: to the nearest multiple of patch size (16) per axis, filled with the volume's minimum intensity value (DivisiblePadd(..., mode="minimum"))
• Intensity normalization: per-volume z-score (NormalizeIntensityd(nonzero=True, channel_wise=True)), plus empty-signal filling (SignalFillEmptyd) - done once ahead of time when building the preprocessed dataset (scripts/preproccess_fomo300k.py), not per-forward-pass
• Augmentations (training only): 3D multi-crop (128^3 global / 48^3 local scaled up to 192^3/64^3 in the high-res phase; crop scale ranges 0.32-1.0 global, 0.05–0.32 local; 8 local crops per sample), random flips. No color jitter (single-channel intensities, not RGB).

Training Hyperparameters

All training hyperparameters can be found in the config.yaml file in this HF repo.