Downloads · 30 days
59
5% of all-time downloads
laion/ViCLIP-L-14-BVD-V-50M-s50M-b32K-WiSE-FT
ViCLIP-L-14-BVD-V-50M-s50M-b32K-WiSE-FT is a video classification model from laion. Use it for the video classification task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
1. Model Details 2. Uses 3. Training Details 4. Evaluation 5. How to Get Started with the Model 6. Citation
Downloads · 30 days
59
5% of all-time downloads
All-time downloads
1.3K
Public
Parameters
428M
3.4 GB on disk
Likes
4
Public
Click a slice to open those files.
.pt1.7 GB · 50%
From the Hugging Face model README
This model is a checkpoint merge between ViCLIP-L-14-BVD-V-50M-s50M-b32K and its image-only initialization checkpoint CLIP-ViT-L-14-DataComp.XL-s13B-b90K with a WiSE-FT coefficient $\alpha = 0.1$.
Model Architecture: ViCLIP with ViT-L/14 vision tower.
HF Transformers: The model is directly usable via AutoModel, AutoTokenizer and AutoVideoProcessor from HF Transformers.
This model is intended for research purposes. We hope it enables researchers to better understand and explore zero-shot video classification, video-text retrieval, and generalisation capabilities of video-language models.
Trained on the BVD-V-55M dataset.
Evaluated using CLIP Benchmark video.
| Kinetics-400 top-1 accuracy | UCF-101 top-1 accuracy | HMDB51 top-1 accuracy | MSR-VTT video/text retrieval recall@1 | MSVD video/text retrieval recall@1 |
|---|---|---|---|---|
| 64.3 | 79.9 | 61.0 | 44.6 / 43.7 | 54.5 / 84.6 |
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer, AutoVideoProcessor
model = AutoModel.from_pretrained("laion/ViCLIP-L-14-BVD-V-50M-s50M-b32K-WiSE-FT", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("laion/ViCLIP-L-14-BVD-V-50M-s50M-b32K-WiSE-FT", trust_remote_code=True)
video_processor = AutoVideoProcessor.from_pretrained("laion/ViCLIP-L-14-BVD-V-50M-s50M-b32K-WiSE-FT", trust_remote_code=True)
video = video_processor("path/to/video.mp4")
labels = ["a person playing basketball", "a person swimming", "a person running"]
text_tokens = tokenizer(labels, padding="max_length", truncation=True, max_length=77, return_tensors="pt")
with torch.no_grad():
v_feat = model.get_video_features(video["pixel_values_videos"])
t_feat = model.get_text_features(text_tokens["input_ids"])
logits = 100.0 * v_feat @ t_feat.t()
probs = logits.softmax(dim=-1)
print("Label probs:", probs)
@misc{laionbvd2026,
title={LAION-BVD: A 10-Million-Hour Open Video Dataset for Multimodal Pre-training},
author={Andreas Hochlehnert and Marianna Nezhurina and Mehdi Cherti and Andrej Radonjic and Thaddäus Wiedemer and Christoph Schuhmann and Romain Beaumont and Wieland Brendel and Bernhard Schölkopf and A. Sophia Koepke and Jenia Jitsev and Matthias Bethge},
year={2026},
eprint={2608.24845},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2608.24845},
}