Downloads · 30 days
30
1% of all-time downloads
calpt/CLIP-ViT-B-32-xlm-roberta-base-laion5B-s13B-b90k
CLIP-ViT-B-32-xlm-roberta-base-laion5B-s13B-b90k is a feature extraction model from calpt. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
CLIP ViT-B/32 xlm roberta base - LAION-5B model converted from OpenCLIP to HuggingFace Transformers.
Downloads · 30 days
30
1% of all-time downloads
All-time downloads
5.8K
Public
Repo size
2.9 GB
Likes
4
Public
Click a slice to open those files.
.bin1.5 GB · 100%
From the Hugging Face model README
CLIP ViT-B/32 xlm roberta base - LAION-5B model converted from OpenCLIP to HuggingFace Transformers.
See https://gist.github.com/calpt/8e3555bd11f1916b5169c8125117e5ee for conversion script and more info.
Model uses custom code. Make sure to pass trust_remote_code=True when loading the model.
Example:
import torch
from PIL import Image
from transformers import AutoModel, AutoFeatureExtractor, AutoTokenizer
model = AutoModel.from_pretrained("calpt/CLIP-ViT-B-32-xlm-roberta-base-laion5B-s13B-b90k", trust_remote_code=True)
processor = AutoFeatureExtractor.from_pretrained("laion/CLIP-ViT-B-32-laion2B-s34B-b79K")
tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-base")
image_input = processor(Image.open("CLIP.png"), return_tensors="pt")
text_input = tokenizer(["a diagram", "a dog", "a cat"], return_tensors="pt", padding=True)
with torch.no_grad():
outputs = model(**image_input, **text_input)
text_probs = (100.0 * outputs.logits_per_image.softmax(dim=-1))
print("Label probs:", text_probs)