Downloads · 30 days
20
22% of all-time downloads
sdoerrich97/mopet_dinov3_booster_derma_derma_blood_oct_organs
mopet_dinov3_booster_derma_derma_blood_oct_organs is a image classification model from sdoerrich97. Use it when you need a label for an image. It is set up for mopet. The card lists the license as mit.
Official weights for MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification (EMA4MICCAI 2026 Workshop). This repo holds the booster-derma variant: DermaMNIST co-trained with three auxil…
Downloads · 30 days
20
22% of all-time downloads
All-time downloads
91
Public
Parameters
7.4M
29.8 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors29.8 MB · 100%
From the Hugging Face model README
Official weights for MoPET: Parameter-Efficient Mixture-of-Experts for Unified
Medical Image Classification (EMA4MICCAI 2026 Workshop). This repo holds the
booster-derma variant: DermaMNIST co-trained with three auxiliary MedMNIST+
datasets (Blood, OCT, OrganS) in a single shared MoPET expert pool, part of the
paper's cross-domain "booster" analysis.
MoPET unifies medical image classification across heterogeneous MedMNIST+ tasks
by adapting a frozen DINOv3 ViT-B/16 backbone with a learned sparse top-k
router over a heterogeneous pool of LoRA + BOFT experts injected into the
attention qkv projections, trained with a DeepSeekMoE-style load-balancing loss
and one lightweight classification head per dataset. Only the experts, routers, and
heads are trained; the backbone stays frozen and is reconstructed from timm at
load time.
| Backbone | vit_base_patch16_dinov3.lvd1689m (DINOv3 ViT-B/16), frozen |
| Adapters | 32 PEFT experts per routed layer (20 LoRA, 12 BOFT), top-12 active per token |
| Routing | learned sparse top-k router over the expert pool, injected into the attention qkv projections |
| Input | 3 x 256 x 256 RGB |
| Heads | one linear classification head per dataset |
| Training data | DermaMNIST, BloodMNIST, OCTMNIST, OrganSMNIST (MedMNIST+, CC BY 4.0) |
| Optimizer | AdamW (decoupled weight decay), lr 1e-3, weight decay 0.0 |
| Schedule | 75 epochs, global batch 128, linear warm-up + cosine annealing, gradient clip 1.0, early stopping patience 10 |
| MoE load balancing | DeepSeekMoE-style auxiliary loss, lambda_load=0.05 |
| Initialization | DINOv3 backbone from timm, frozen; only experts, routers and heads are trained |
| Trainable parameters | ~7.4M (experts, routers, per-dataset heads; the frozen backbone is reconstructed from timm at load time) |
These are the lab's shared training-recipe values (see experiments/utils/training.py).
The full multi-task loop is in the repo; a runnable demo is examples/training.ipynb.
The target of this configuration is DermaMNIST; it is co-trained with the three auxiliary datasets to raise target accuracy. A single forward pass routes the input through the shared expert pool and emits the per-dataset head logits.
mopet package supports adding a new dataset head and
co-training it against the existing expert pool (see the repo's experiments/).reproducibility.py).pip install mopet-moe
(B, 3, 256, 256).examples/inference.ipynb.from mopet import create_model
model = create_model(weights="booster-derma").eval()
# ... run inference; see the repo's examples/ notebooks.
The mopet package resolves this repo automatically via its PUBLISHED_MODELS map,
downloading model.safetensors through huggingface_hub.
Accuracy (ACC) and area under the ROC curve (AUC), per the paper.
From the paper: co-training a data-constrained target dataset with a hand-picked pool of auxiliary datasets raises target accuracy (81.58% -> 83.58% on the data-constrained target, averaged across the paper's booster configurations). This is one of the two booster configurations reported in the paper (Table 3); see the paper for the exact per-dataset numbers for this pool.
@article{doerrich2026mopet,
title={MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification},
author={Sebastian Doerrich and Daniel W{\"u}rtinger and Francesco {Di Salvo} and Shyam Nandan Rai and Christian Ledig},
year={2026},
eprint={2607.29462},
archivePrefix={arXiv},
primaryClass={eess.IV},
url={https://arxiv.org/abs/2607.29462},
}
For questions or issues, please open an issue in the GitHub repository.