Downloads · 30 days
1.9K
1% of all-time downloads
Qwen3.5-9B-NVFP4
apolo13x/Qwen3.5-9B-NVFP4
Qwen3.5-9B-NVFP4 is a image-text-to-text model from apolo13x. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is a quantized version of Qwen/Qwen3.5-9B. This model accepts text and images as inputs and generates text as outputs. The weights and activations were quantized to FP4 using llm-compressor with 512 calibration s…
All-time downloads
360K
Public
Parameters
7.5B
12.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors12.4 GB · 100%
Parameter types
How the weights are stored.
BF164.8B · 64%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.5-9B-NVFP4
This is a quantized version of Qwen/Qwen3.5-9B. This model accepts text and images as inputs and generates text as outputs. The weights and activations were quantized to FP4 using llm-compressor with 512 calibration samples from nvidia/Nemotron-Post-Training-Dataset-v2, reducing the model size from 18.0 GB to 11.5 GB (~1.6x reduction) while maintaining 97.3% average accuracy recovery.
Quantization Details
- Scheme: NVFP4
- Calibration: 512 samples (256 reasoning-on + 256 reasoning-off) from Nemotron-Post-Training-Dataset-v2
- Max sequence length: 4096
Inference
This model is supported in vLLM 0.17.0. To serve the model:
vllm serve Kbenkhaled/Qwen3.5-9B-NVFP4 \
--reasoning-parser qwen3 \
--enable-prefix-caching
Evaluation
Evaluated with lm-evaluation-harness, 0-shot, thinking mode ON.
| Benchmark | Qwen3.5-9B | Qwen3.5-9B-NVFP4 (this model) | Recovery |
|---|---|---|---|
| GPQA Diamond | 78.79% | 74.24% | 94.2% |
| IFEval | 94.48% | 92.69% | 98.1% |
| MMLU-Redux | 91.80% | 91.39% | 99.6% |
| Average | 88.36% | 86.11% | 97.3% |
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Mar 3, 2026
- Updated
- Mar 11, 2026