vit_giant_patch14_reg4_224.h-optimus-v0

Model card for vit_giant_patch14_reg4_224.h-optimus-v0

Model Details

• Model Type: Feature backbone
• Model Stats:
  • Params: 1.13B (giant)
  • Image size: 224 x 224 x 3
  • Patch size: 14 x 14 x 3
  • Registers: 4
• Repository: github.com:bioptimus/releases
• Original Weights: https://public-bioptimus-eu-west-3.s3.eu-west-3.amazonaws.com/h-optimus-v0/checkpoint.pth

Model Usage

Image Embeddings

from PIL import Image
import torch
import timm

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_giant_patch14_reg4_224.h-optimus-v0",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

img = Image.open(...)
data = transforms(img).unsqueeze(0) # input is a (batch_size, num_channels, img_size, img_size) shaped tensor
output = model(data)  # output is a (batch_size, num_features) shaped tensor

Citation

@software{hoptimus0,
  title  = {H-optimus-0},
  author = {Saillard, Charlie and Jenatton, Rodolphe and Llinares-López, Felipe and Mariet, Zelda and Cahané, David and Durand, Eric and Vert, Jean-Philippe},
  url    = {https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0},
  year   = {2024},
}