Downloads · 30 days
15
24% of all-time downloads
birder-project/rope_vit_reg8_b14_nps_avg_capi-dino-bio
rope_vit_reg8_b14_nps_avg_capi-dino-bio is a image feature extraction model from birder-project. Use it for the image feature extraction task on the model card, and read the license before you ship it in a product. It is set up for birder. The card lists the license as apache-2.0.
A RoPE ViT Reg8 B/14 image encoder with average pooling, pretrained using CAPI-DINO on natural biological images. This model has not been fine-tuned for a specific classification task and is intended to be used as a g…
Downloads · 30 days
15
24% of all-time downloads
All-time downloads
63
Public
Repo size
343 MB
Likes
1
Public
Click a slice to open those files.
.pt343 MB · 100%
From the Hugging Face model README
A RoPE ViT Reg8 B/14 image encoder with average pooling, pretrained using CAPI-DINO on natural biological images. This model has not been fine-tuned for a specific classification task and is intended to be used as a general-purpose feature extractor or a backbone for downstream tasks like object detection, segmentation, or custom classification.
Model Type: Image encoder and detection backbone
Model Stats:
Dataset: Trained on a diverse dataset of approximately 31M images, including:
Papers:
import birder
from birder.inference.classification import infer_image
# Option 1: manual setup (more control over preprocessing)
net, model_info = birder.load_pretrained_model("rope_vit_reg8_b14_nps_avg_capi-dino-bio", inference=True)
# Get the image size the model was trained on
size = birder.get_size_from_signature(model_info.signature)
# Create an inference transform
transform = birder.classification_transform(size, model_info.rgb_stats)
# Option 2: helper (quick start with default preprocessing)
net, model_info, transform = birder.load_pretrained_model_and_transform("rope_vit_reg8_b14_nps_avg_capi-dino-bio", inference=True)
image = "path/to/image.jpeg" # or a PIL image
out, embedding = infer_image(net, image, transform, return_embedding=True)
# embedding is a NumPy array with shape of (1, 768)
from PIL import Image
import birder
net, model_info, transform = birder.load_pretrained_model_and_transform("rope_vit_reg8_b14_nps_avg_capi-dino-bio", inference=True)
image = Image.open("path/to/image.jpeg")
features = net.detection_features(transform(image).unsqueeze(0))
# features is a dict (stage name -> torch.Tensor)
print([(k, v.size()) for k, v in features.items()])
# Output example:
# [('stage1', torch.Size([1, 768, 16, 16]))]
@misc{dosovitskiy2021imageworth16x16words,
title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
author={Alexey Dosovitskiy and Lucas Beyer and Alexander Kolesnikov and Dirk Weissenborn and Xiaohua Zhai and Thomas Unterthiner and Mostafa Dehghani and Matthias Minderer and Georg Heigold and Sylvain Gelly and Jakob Uszkoreit and Neil Houlsby},
year={2021},
eprint={2010.11929},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2010.11929},
}
@misc{heo2024rotarypositionembeddingvision,
title={Rotary Position Embedding for Vision Transformer},
author={Byeongho Heo and Song Park and Dongyoon Han and Sangdoo Yun},
year={2024},
eprint={2403.13298},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2403.13298},
}
@misc{darcet2024visiontransformersneedregisters,
title={Vision Transformers Need Registers},
author={Timothée Darcet and Maxime Oquab and Julien Mairal and Piotr Bojanowski},
year={2024},
eprint={2309.16588},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2309.16588},
}
@misc{darcet2025clusterpredictlatentpatches,
title={Cluster and Predict Latent Patches for Improved Masked Image Modeling},
author={Timothée Darcet and Federico Baldassarre and Maxime Oquab and Julien Mairal and Piotr Bojanowski},
year={2025},
eprint={2502.08769},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2502.08769},
}