dinov2-small-droid

dinov2-small-droid

This is a DINOv2-small vision backbone adapted on robot-manipulation images from the DROID dataset. It starts from facebook/dinov2-small and preserves the standard Hugging Face Transformers DINOv2 architecture.

• exported weights: teacher backbone
• training step: 50,000
• input size: 518
• patch size: 14
• hidden size: 384
• training-only projection heads and optimizer state: not included

Usage

from PIL import Image
import torch
from transformers import AutoImageProcessor, AutoModel

model_id = "SakikoTogawa/dinov2-small-droid"
processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id).eval()

image = Image.open("example.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.inference_mode():
    outputs = model(**inputs)

cls_feature = outputs.last_hidden_state[:, 0]   # [batch, 384]
patch_features = outputs.last_hidden_state[:, 1:]

For cosine similarity or nearest-neighbor retrieval:

cls_feature = torch.nn.functional.normalize(cls_feature, dim=-1)

Intended use and limitations

The model is intended as an image-feature backbone for downstream robotics experiments such as frozen-feature evaluation, linear probing, retrieval, or further fine-tuning. The adaptation objective is self-supervised and does not guarantee improvement on every downstream task. Compare against the original facebook/dinov2-small using an episode-disjoint evaluation split.

Training data

DROID contains diverse robot-manipulation trajectories collected across multiple scenes and tasks. Consult the DROID project page for dataset details and terms.