Downloads · 30 days
78
11% of all-time downloads
Qwen3-VL-235B-A22B-Instruct-NVFP4
GadflyII/Qwen3-VL-235B-A22B-Instruct-NVFP4
Qwen3-VL-235B-A22B-Instruct-NVFP4 is a image-text-to-text model from GadflyII. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
NVFP4 (4-bit floating point) quantized version of Qwen/Qwen3-VL-235B-A22B-Instruct, optimized for NVIDIA Blackwell GPUs.
All-time downloads
708
Public
Parameters
133B
135 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors135 GB · 100%
Parameter types
How the weights are stored.
U8117B · 88%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-VL-235B-A22B-Instruct-NVFP4
NVFP4 (4-bit floating point) quantized version of Qwen/Qwen3-VL-235B-A22B-Instruct, optimized for NVIDIA Blackwell GPUs.
Model Details
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-VL-235B-A22B-Instruct |
| Parameters | 235B (22B active per token) |
| Architecture | MoE (Mixture of Experts) + Vision |
| Quantization | NVFP4 (weights & activations) |
| Original Size | ~471 GB (BF16) |
| Quantized Size | ~127 GB |
| Compression | ~3.7x |
Quantization Details
- Tool: llmcompressor
- Scheme: NVFP4 (4-bit floating point with block-wise scaling)
- Calibration: 512 samples, 4096 sequence length
- Pipeline: Sequential (layer-by-layer for memory efficiency)
- Preserved layers: embeddings, lm_head, vision encoder, MoE gates/routers
Hardware Requirements
- Minimum VRAM: ~130 GB (TP=2 recommended)
- Optimized for: NVIDIA Blackwell (SM120) - RTX 5090, RTX PRO 6000
- Also works on: NVIDIA Hopper (SM90) - H100, H200
Usage with vLLM
# Environment variables for Blackwell GPUs
export VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8_CUTLASS=1
export VLLM_FLASH_ATTN_VERSION=2
# Serve with tensor parallelism
vllm serve GadflyII/Qwen3-VL-235B-A22B-Instruct-NVFP4 \
--tensor-parallel-size 2 \
--trust-remote-code \
--max-model-len 32768
Performance
Tested on 2x NVIDIA RTX PRO 6000 Blackwell (192GB VRAM total) with vLLM 0.13.0:
| Metric | Value |
|---|---|
| Single Request Decode | 57 tokens/s |
| Batch Throughput (8 req) | 277 tokens/s |
| Memory Usage | ~128 GB (64 GB/GPU) |
| Model Load Time | ~30s |
Acknowledgments
- Original model by Qwen Team
- Quantization using vLLM llmcompressor
License
Apache 2.0 (same as base model)
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_vl_moe
- License
- apache-2.0
- Created
- Dec 30, 2025
- Updated
- Jan 26, 2026