deepseek_vit_412m_align.deepseek_v4_flash_vision_exp

Model card for deepseek_vit_412m_align.deepseek_v4_flash_vision_exp

NOTE: This checkpoint is a native timm remap of the original vision weights, with no additional training. It contains no language-model weights or trained image-classification head.

A DeepSeek ViT image feature model extracted from DeepSeek-V4-Flash-Vision-Exp. This is the classifier-ready wrapper retaining the native spatial aligner and LLM-width projection, followed by average pooling and affine-free RMSNorm.

Model Notes

• The backbone uses 14×14 patches, SwiGLU MLPs, RMSNorm and axial 2D RoPE, with no learned absolute position embeddings. The original linear patch projection is reshaped into a Conv2d without changing its computation.
• The native aligner groups 3×3 patch tokens in channel-major order and uses a two-layer GELU MLP to project to the source LLM width. Incomplete groups are zero-padded on the bottom/right. It is retained in _enc and _align variants and omitted from the plain classifier.
• RGB inputs use mean=(0.5, 0.5, 0.5) and std=(0.5, 0.5, 0.5), matching the original. The default timm evaluation transform uses crop_mode="border", crop_pct=1.0 and bicubic resizing to preserve aspect ratio on a fixed, gray-padded canvas. The original processor selects variable canvas dimensions and uses gray 127 padding; timm uses gray 128.
• Rectangular inputs are supported. Dimensions must be divisible by 14 by default. Pass dynamic_img_pad=True at model creation to zero-pad normalized inputs on the bottom/right to a patch-size multiple. This does not reproduce the original adaptive resize policy.
• forward_features() returns projected NLC tokens from the aligner. encoder.forward_features() returns final-RMSNorm NHWC backbone features. forward() returns pooled image embeddings until a classification head is added.
• Intermediate backbone maps are available through forward_intermediates() and features_only=True; these do not include the aligner. Use norm=True to apply the encoder's final RMSNorm to intermediate maps.

Model Details

• Model Type: Image Feature Encoder
• Model Stats:
  • Params (M): 466.4
  • GMACs: 368.5
  • Activations (M): 611.8
  • Image size: 392 x 392
• Source revision: 6821d6ad3681a4b137b066b76094fa82ebd0a380
• License source: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/blob/6821d6ad3681a4b137b066b76094fa82ebd0a380/LICENSE
• Original code: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/blob/6821d6ad3681a4b137b066b76094fa82ebd0a380/inference/vision.py
• Original preprocessing: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/blob/6821d6ad3681a4b137b066b76094fa82ebd0a380/inference/image_processor.py
• Original: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
• License: MIT
• Backbone width: 1024
• Projection width: 4096
• Papers:
  • DeepSeek-V4-Flash-Vision-Exp: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
  • PyTorch Image Models: https://github.com/huggingface/pytorch-image-models

Model Usage

Image Features

import torch
import timm
from PIL import Image

model = timm.create_model('hf-hub:timm/deepseek_vit_412m_align.deepseek_v4_flash_vision_exp', pretrained=True).eval()
data_config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**data_config, is_training=False)
image = Image.open('image.jpg').convert('RGB')
x = transform(image).unsqueeze(0)

with torch.inference_mode():
    output = model(x)  # (1, 4096): image embeddings
    features = model.forward_features(x)  # (1, 100, 4096): projected spatial tokens (NLC)

Intermediate Feature Maps

with torch.inference_mode():
    maps = model.forward_intermediates(
        x, indices=3, norm=True, output_fmt='NCHW', intermediates_only=True,
    )
for feature_map in maps:
    print(feature_map.shape)  # (1, 1024, 28, 28)

Classification Fine-tuning

model = timm.create_model(
    'hf-hub:timm/deepseek_vit_412m_align.deepseek_v4_flash_vision_exp', pretrained=True, num_classes=45,
)
logits = model(x)  # (1, 45)

The new linear head is randomly initialized and must be trained on your target dataset.

Citation

@misc{deepseekai2026deepseekv4flashvisionexp,
  title={DeepSeek-V4-Flash-Vision-Exp},
  author={DeepSeek-AI},
  year={2026},
  url={https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp},
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}