DeepSeek-V4-Flash-Vision-NVFP4

[!TIP] Looking for the newer DeepSeek V4 Flash 0731 backbone? Use the 0731 NVFP4 vision package.

DeepSeek V4 Flash Vision (NVFP4)

DeepSeek V4 Flash with sight. A vision-language development checkpoint that connects DeepSeek's reasoning and agentic model to the MoonViT vision encoder from Kimi-K2.6 through a trained, routing-aware PatchMerger projector.

The text backbone and vision tower remain frozen. The only newly trained parameters are the 40,119,040-parameter projector that merges each 2x2 group of MoonViT patches and maps the resulting 4608-dimensional representation into DeepSeek's 4096-dimensional token space. Original text routing IDs are preserved; image positions receive deterministic routing IDs from a fixed 64-ID palette.

Why vision at WebBrain

At WebBrain, we build browser agents that need to understand the visual state of the web—not just extracted text. Screenshots, charts, dashboards, rich editors, and the location and appearance of controls are part of real browser work, so vision is a practical product requirement.

In our American–Chinese open-model frontier benchmark, DeepSeek V4 Flash stood out as a very strong model and the cheapest to run in its class, but the upstream checkpoint is text-only. This project adds a basic MoonViT vision bridge while keeping both the language backbone and vision tower frozen.

[!IMPORTANT] The pinned NVFP4 text backbone, frozen MoonViT tower, and trained 100K-example projector are complete and verified. Reference BF16 multimodal inference has passed end-to-end and KV-cache parity checks. The repository now includes a custom, commit-pinned SGLang processor, routing bridge, source patch, and B200 launch recipe. The pinned B200 runtime has passed full NVFP4 loading and server startup plus two live image-generation smoke tests. Broader text-only parity and production validation are still required. This is not a drop-in checkpoint for a stock text-only server.

Build status

• Pin the upstream NVIDIA DeepSeek V4 Flash NVFP4 revision.
• Copy and fingerprint-verify all 54 backbone files (168.30 GB).
• Extract and validate the frozen MoonViT component.
• Pass real BF16 H200 forward/backward, overfit, and production-mix calibration gates.
• Materialize and cache the 100,000-example MoonViT training set.
• Finish the 100,000-example MoonViT projector run.
• Add and fingerprint-verify the frozen MoonViT tower and final projector checkpoint.
• Pass reference BF16 image inference and KV-cache/full-prefix token parity.
• Assemble multimodal configuration, processor, routing bridge, and serving integration.
• Pass pinned B200 NVFP4 shard loading and server-startup gates.
• Pass two live image-response smoke tests on the pinned B200 runtime.
• Complete text-only parity and broader production validation.

Provenance

The packaged text backbone is copied from nvidia/DeepSeek-V4-Flash-NVFP4 at immutable revision e3cd60e7de98e9867116860d522499a728de1cf9. All 54 copied files were checked against their upstream Git blob or LFS SHA-256 fingerprints after upload.

Projector training uses a frozen BF16 reconstruction of DeepSeek V4 Flash because the released inference quantization kernels do not provide the input-gradient path needed to train through a frozen language model. MoonViT is pinned to Kimi-K2.6 revision 7eb5002f6aadc958aed6a9177b7ed26bb94011bb. This package pairs the trained projector with the verified NVIDIA NVFP4 backbone above. Reference BF16 parity and live B200 image generation have been validated; broader text-only parity and production behavior are not claimed.

Usage

Stock text-only engines do not know how to combine these artifacts. Use the custom, version-pinned SGLang package and wrapper in docs/SGLANG_DEPLOYMENT.md. The B200 startup profile is selected explicitly and keeps the first correctness run conservative:

export DEEPSEEK_VISION_MODEL_PATH=/models/deepseek-v4-flash-vision
export DEEPSEEK_VISION_PYTHONPATH="$DEEPSEEK_VISION_MODEL_PATH/sglang_ext"
export DEEPSEEK_VISION_KERNEL_PROFILE=blackwell-native
export DEEPSEEK_VISION_TP=4
scripts/launch_sglang_moonvit.sh

The wrapper uses the native flashinfer_trtllm dense backend and flashinfer_trtllm_routed MoE backend on the documented B200 image. Two live image-response smoke tests passed on this profile; do not treat that bounded smoke result as production validation.

Method credit

The overall construction and model-card approach was inspired by Baseten's GLM-5.2-Vision-NVFP4: keep the text backbone and MoonViT tower frozen, train a compact PatchMerger projector between them, and publish provenance and hardware constraints explicitly. Credit to the Baseten team for demonstrating this practical recipe.

This project adds a DeepSeek-specific routing bridge so mixed text/image embeddings preserve hash-routed text behavior. It does not reuse Baseten model weights, benchmark results, or deployment artifacts.

License

The redistributed DeepSeek V4 Flash NVFP4 backbone remains subject to the included MIT license and upstream notices. MoonViT assets added later will remain subject to the Kimi-K2.6 Modified MIT terms. Newly trained projector artifacts will be documented with their applicable terms when uploaded.

Acknowledgements

Built on DeepSeek AI's DeepSeek V4 Flash, NVIDIA's NVFP4 checkpoint, and Moonshot AI's Kimi-K2.6, with the vision-attachment method inspired by Baseten's GLM-5.2-Vision-NVFP4. These teams were not involved in this experimental development checkpoint; please do not direct issues with this repository to them.

Want this model on your inference provider?

Ask your inference provider—such as OpenRouter or another OpenAI-compatible managed service—to deploy this exact repository with its multimodal processor and serving plugin. Deploying only the upstream text backbone will not enable image input.

Experimental status, roadmap, and get involved

[!CAUTION] Experimental vision adapter. This is a working experimental adapter with basic end-to-end SGLang image generation verified on NVIDIA B200. That bounded smoke test is not a broad quality benchmark: fine-grained OCR, small-object or control identification, GUI grounding, and hallucination calibration remain limited. If community interest warrants further investment, the roadmap is larger and more diverse datasets, higher-resolution OCR/UI examples, and broader parameter-efficient tuning. Do not use this model as the sole decision source for safety-critical automation. Interested in contributing evaluation or training data, sponsoring compute, or working with us as a design partner? Tell us here.