last-vit-clip-vitb16

LAST-ViT CLIP ViT-B/16 (Text-Supervised)

Vision encoder from OpenCLIP ViT-B/16 with LAST-ViT pooling. Converted to HuggingFace Transformers format from the LAST-ViT openai_b_16 checkpoint.

This is the vision encoder only (no text encoder). Use for image embedding extraction or as a backbone for vision tasks.

Model Details

• Architecture: ViT-B/16 (Vision Transformer Base, patch size 16×16)
• Training scenario: Text-supervised (CLIP)
• Hidden size: 768
• Num layers: 12
• Num attention heads: 12
• Image size: 224×224

Usage

from transformers import ViTModel, ViTImageProcessor
from PIL import Image

model = ViTModel.from_pretrained("BiliSakura/last-vit-clip-vitb16")
processor = ViTImageProcessor.from_pretrained("BiliSakura/last-vit-clip-vitb16")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
outputs = model(**inputs)

# Image embeddings (CLS token)
embeddings = outputs.last_hidden_state[:, 0, :]  # (batch, 768)

References

• Repository: ChengShiest/LAST-ViT
• CLIP training: mlfoundations/open_clip
• Original weights: openai_b_16.pt