vit_small_patch16_224.lunit_dino

Model card for vit_small_patch16_224.lunit_dino

A Vision Transformer (ViT) image classification model.
Trained on 33M histology patches from various pathology datasets.

Model Details

• Model Type: Feature backbone
• SSL Method: DINO
• Model Stats:
  • Params (M): 21.1
  • Image sizes: 224 × 224 x 3
• Papers:
  • Benchmarking Self-Supervised Learning on Diverse Pathology Datasets: https://arxiv.org/abs/2212.04690
• Datasets:
  • BACH
  • CRC
  • MHIST
  • PatchCamelyon
  • CoNSeP
• Original: https://github.com/lunit-io/benchmark-ssl-pathology
• License: lunit-non-commercial

Model Usage

Image Embeddings

from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://github.com/owkin/HistoSSLscaling/raw/main/assets/example.tif"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_small_patch16_224.lunit_dino",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

Citation

@inproceedings{kang2022benchmarking,
  author    = {Kang, Mingu and Song, Heon and Park, Seonwook and Yoo, Donggeun and Pereira, Sérgio},
  title     = {Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023},
  pages     = {3344-3354}
}