Downloads · 30 days
19
12% of all-time downloads
Qwen3-VL-4B-Instruct-W4A16-BF16Vision
useful-quants/Qwen3-VL-4B-Instruct-W4A16-BF16Vision
Qwen3-VL-4B-Instruct-W4A16-BF16Vision is a image-text-to-text model from useful-quants. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Ampere-friendly serving build of Qwen/Qwen3-VL-4B-Instruct. Text-side linears are compressed-tensors W4A16. The vision tower and multimodal connector stay BF16.
All-time downloads
160
Public
Parameters
4.5B
3.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.5 GB · 100%
Parameter types
How the weights are stored.
I323.6B · 81%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-VL-4B-Instruct W4A16 BF16-Vision
Ampere-friendly serving build of Qwen/Qwen3-VL-4B-Instruct.
Text-side linears are compressed-tensors W4A16. The vision tower and multimodal connector stay BF16.
Use
vllm serve useful-quants/Qwen3-VL-4B-Instruct-W4A16-BF16Vision \
--served-model-name Qwen3-VL-4B-Instruct-W4A16-BF16Vision \
--trust-remote-code \
--dtype bfloat16 \
--quantization compressed-tensors \
--max-model-len 4096 \
--limit-mm-per-prompt '{"image":1}' \
--allowed-local-media-path / \
--enforce-eager
Note: --enforce-eager is recommended in some vLLM versions to avoid buffer issues with deepstack tokens.
Smoke test
python verify.py --url http://localhost:8000/v1/chat/completions
Notes
- Built for RTX 30xx/40xx Ampere cards.
- Vision stays BF16 on disk and at runtime.
- The repo includes the calibration corpus used for quantization.
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_vl
- License
- apache-2.0
- Created
- May 7, 2026
- Updated
- May 7, 2026