clip-vit-base-32

Learning Transferable Visual Models From Natural Language Supervision

https://arxiv.org/abs/2103.00020

Lucid port of https://huggingface.co/openai/clip-vit-base-patch32, converted to Lucid-native safetensors.

Available weights

Usage

import lucid
import lucid.models as models
from lucid.models.weights import ClipWeights

# default tag
model = models.clip_vit_base_32(pretrained=True)

# explicit tag (enum or string)
model = models.clip_vit_base_32(weights=ClipWeights.OPENAI_WIT400M)
model = models.clip_vit_base_32(pretrained="OPENAI_WIT400M")

# feed token ids (tokenize with the matching lucid.utils.tokenizer)
input_ids = lucid.tensor([[101, 7592, 2088, 102]], dtype=lucid.int64)
out = model(input_ids)
hidden = out.last_hidden_state  # (B, T, hidden_size)

Conversion

Converted from https://huggingface.co/openai/clip-vit-base-patch32 via python -m tools.convert_weights CLIP --tag OPENAI_WIT400M. Key mapping + numerical parity verified against the source.

License

mit — inherited from the original weights.

Citation

@inproceedings{radford2021learning,
  title={Learning Transferable Visual Models From Natural Language Supervision},
  author={Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya},
  booktitle={ICML},
  pages={8748--8763},
  year={2021}
}