Downloads · 30 days
202
1% of all-time downloads
shunyang90/Qwen3-Omni-30B-A3B-Instruct-NVFP4
Qwen3-Omni-30B-A3B-Instruct-NVFP4 is a any-to-any model from shunyang90. Use it for the any-to-any task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Pre-quantized NVFP4 version of Qwen/Qwen3-Omni-30B-A3B-Instruct for deployment on NVIDIA Blackwell GPUs (RTX 5090, B100, B200).
Downloads · 30 days
202
1% of all-time downloads
All-time downloads
23.7K
Public
Parameters
21.7B
89.2 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors25.9 GB · 100%
How the weights are stored.
U815.5B · 72%
From the Hugging Face model README
Pre-quantized NVFP4 version of Qwen/Qwen3-Omni-30B-A3B-Instruct for deployment on NVIDIA Blackwell GPUs (RTX 5090, B100, B200).
| Component | Precision | Quantized? |
|---|---|---|
| Thinker MoE experts (gate_up_proj, down_proj) | NVFP4 packed uint8 | Yes — pre-quantized |
| Thinker attention (q/k/v/o_proj) | NVFP4 (calibrated) | Yes |
| Thinker lm_head | BF16 | No |
| Thinker MoE router gates | BF16 | No |
| Audio Encoder | BF16 | No |
| Vision Encoder | BF16 | No |
| Talker (MoE) | BF16 | No |
| Code2Wav | BF16 | No |
MoE expert weights are stored as:
gate_up_proj: packed uint8 (2× FP4 values per byte)gate_up_proj_scale: float8_e4m3fn (per-block-of-16 FP8 scale)gate_up_proj_scale_2: bfloat16 (global per-tensor scale)down_proj{
"quant_algo": "NVFP4",
"quant_method": "modelopt",
"group_size": 16,
"weights": {"num_bits": 4, "type": "float"},
"input_activations": {"num_bits": 8, "type": "float"},
"kv_cache": {"num_bits": 8, "type": "float"}
}
| Config | Checkpoint Size | Notes |
|---|---|---|
| BF16 (original) | ~60 GB | Full precision |
| FP8 (ModelOpt) | ~40 GB | Online quantization |
| NVFP4 (this model) | ~27 GB | Pre-quantized FP4 experts |
| Purpose | Hardware | Notes |
|---|---|---|
| Inference | RTX 5090, B100, B200 (Blackwell SM 100+) | Native FP4 hardware support required |
| Inference (not supported) | RTX 4090, A100, H100 | No FP4 hardware — use FP8 instead |
mtq.quantize with calibrationNVFP4QTensor.quantize() (fused expert tensors not supported by mtq.quantize)This is a quantized version of Qwen/Qwen3-Omni-30B-A3B-Instruct. Please refer to the original model card for full capabilities, benchmarks, and usage documentation.
Apache License 2.0 (same as the original model)