Downloads · 30 days
11
4% of all-time downloads
VideoChat2_HD_Infinity_Mistral_7B
Rihong/VideoChat2_HD_Infinity_Mistral_7B
VideoChat2_HD_Infinity_Mistral_7B is a video-text-to-text model from Rihong. Use it for the video-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers.
HuggingFace-compatible model files for VideoChat2-Infinity (Mistral-7B), a long video understanding model that extends VideoChat2-HD with infinite-length video processing via continuous long-term attention.
All-time downloads
305
Public
Parameters
7.7B
16.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.4 GB · 100%
Parameter types
How the weights are stored.
F167.2B · 94%
Base models
Model card
From the Hugging Face model README
Infinite-VideoChat2
HuggingFace-compatible model files for VideoChat2-Infinity (Mistral-7B), a long video understanding model that extends VideoChat2-HD with infinite-length video processing via continuous long-term attention.
Model Architecture
- Vision Encoder – UMT-L (ViT, 1024-dim, 24 layers)
- Q-Former – bridges visual features to the LLM with 32 + 64 query tokens
- LLM – Mistral-7B-Instruct-v0.2
- Long-Term Attention – Gibbs-sampling-based continuous attention over basis functions for unbounded temporal context
File Overview
| File | Description |
|---|---|
config.json | HuggingFace AutoConfig configuration |
configuration_videochat2.py | Custom PretrainedConfig subclass (Config) |
videochat2_it_hd_mistral.py | Main model class (VideoChat2_it_hd_mistral) |
blip2.py | BLIP-2 base class for vision-language bridging |
vit.py | Vision Transformer (UMT-L) implementation |
Qformer.py | Q-Former module |
basis_functions.py | Basis functions (Power, Sine, Cosine, Gaussian, Rectangular) for long-term attention |
long_term_attention_gibbs.py | Long-term attention with Gibbs sampling |
model-*.safetensors | Model weights (sharded, 4 parts) |
.gitignore | Ignores model weight files |
Usage
The model is registered with HuggingFace auto_map, so it can be loaded directly:
from transformers import AutoConfig, AutoModel
config = AutoConfig.from_pretrained("Rihong/VideoChat2_Infinity_Mistral_7B_hf", trust_remote_code=True)
model = AutoModel.from_pretrained("Rihong/VideoChat2_Infinity_Mistral_7B_hf", trust_remote_code=True)
To upload model to HuggingFace, just run:
hf upload Rihong/VideoChat2_Infinity_Mistral_7B_hf ./lmms_eval/baselines/infty_videochat2/
Key Hyperparameters
| Parameter | Default | Description |
|---|---|---|
num_basis | 256 | Number of basis functions for long-term attention |
tau | 0.75 | Temperature for Gibbs sampling |
alpha | 0.75 | Mixing coefficient |
sticky | true | Enable sticky memories |
hd_num | 6 | Number of high-definition crops |
local_size | 224 | Local crop resolution |
References
- Task
- Video-Text-to-Text
- Library
- transformers
- Languages
- en
- Created
- Mar 2, 2026
- Updated
- Mar 3, 2026