Dual-IFM

Dual-IFM: pretrained models

Pretrained model weights from "Towards Interpretable Foundation Models for Retinal Fundus Images". Code: berenslab/interpretable_FM.

This repo hosts the pretrained SSL models only (SimCLR / t-SimCNE / t-SimCNE-2D). To finetune downstream classifiers on top of these weights see the GitHub repo.

Model description

Dual-IFM is an interpretable foundation model for retinal fundus images that combines local explanations with a direct visualization of the representation space:

• Local explanations are provided by the BagNet backbone: its small receptive fields produce class evidence maps that indicate which regions of the image contributed to a given prediction, without relying on post-hoc attribution methods.
• Visualizations of the embedding space are provided by a projection layer that maps images into a 2D approximation of the representation space, without re-fitting a non-parametric method such as t-SNE, providing a dataset-level view of the learned structure.

Model variants

One shared repo, one subfolder per variant, all pretrained on a combined EyePACS + AREDS + UKB fundus dataset ("all"). All subfolder names follow the format: <method>-<backbone>-<image_size>, all models were trained at a 256x256 resolution.

tsimcne2d-bagnet33-256 is the main Dual-IFM checkpoint: trained with t-SimCNE using SimCLR cosine-similarity loss for stage one before switching to the standard t-SimCNE Euclidean/Cauchy-similarity loss for the last two stages.

SimCLR checkpoints keep the standard 128-dim contrastive projection head; t-SimCNE and t-SimCNE-2D mutate the projector's last layer down to 2D during training so the embeddings can be plotted directly, without a separate dimensionality reduction step.

Training data

Pretrained on images from three color fundus photography (CFP) datasets, preprocessed (cropped to a centered circle) and filtered for quality, totalling 802,360 images:

Usage

Environment setup

This project uses uv for fast Python environment and dependency management, but the dependencies can be installed into any environment with pip. The hub extra installs huggingface_hub and safetensors, needed to load the weights from the HF Hub.

Clone the GitHub repository:

git clone https://github.com/berenslab/interpretable_FM.git
cd interpretable_FM

Option 1 — Using uv (recommended)

uv sync --extra hub
source .venv/bin/activate
uv pip install -e .

Option 2 — Using pip

pip install -e ".[hub]"

Loading the model

from dual_ifm.utils.hf_hub import DualIFM

model = DualIFM.from_pretrained("CamilaR20/Dual-IFM", subfolder="tsimcne2d-bagnet33-256")

For more usage examples refer to the GitHub repository.

Evaluation

Dual-IFM was evaluated via linear probing and fine-tuning on held-out test sets of the pretraining datasets (EyePACS, AREDS) as well as out-of-distribution datasets: APTOS, IDRiD, DeepDRiD, and Messidor-2 (diabetic retinopathy grading), Glaucoma fundus and PAPILA (glaucoma detection), and FIVES (multi-disease classification).

It performs comparably to RETFound (ViT-Large), while using roughly 16× fewer parameters (18.3M for BagNet-33 vs. 303.3M for RETFound).

Limitations

This is a research model, not a clinically validated diagnostic tool. It has not been evaluated for deployment in clinical setting, and performance on populations or imaging devices not represented in the pretraining data (EyePACS, AREDS, UKB) is not guaranteed.

Citation

@misc{mensah2026dualifm,
  title={Towards Interpretable Foundation Models for Retinal Fundus Images},
  author={Mensah, Samuel Ofosu and Roa, Camila and Djoumessi, Kerol and Berens, Philipp},
  year={2026},
  eprint={2603.18846},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  doi={10.48550/arXiv.2603.18846}
}