Downloads · 30 days
38
11% of all-time downloads
Qwen3-4B-NVFP4
vistralis/Qwen3-4B-NVFP4
Qwen3-4B-NVFP4 is a text generation model from vistralis. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
NVFP4 (W4A4) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
All-time downloads
338
Public
Parameters
2.8B
3.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors3.6 GB · 100%
Parameter types
How the weights are stored.
U81.8B · 64%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-4B-NVFP4
NVFP4 (W4A4) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
Overview
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-4B |
| Parameters | 4.41B |
| Quantization | NVFP4 (W4A4) |
| Format | nvfp4-pack-quantized via compressed-tensors |
| Tool | llm-compressor |
| Disk Size | ~3.4 GB |
| VRAM | ~2.2 GB |
Intended Use
Quantized text encoder for Flux 2 Klein 4B image generation pipelines. Architecturally identical to the Klein 4B text encoder.
Quantization Details
- Scheme: NVFP4 — 4-bit weights with 4-bit activations, FP8 block scales, FP32 global scale
- Targets: All
Linearlayers (excludinglm_head) - Calibration: 256 samples, sequential pipeline with CPU offloading
- Group Size: 16
- Scale Dtype:
float8_e4m3fn
Parity
Verified against BF16 baseline:
| Scope | Cosine Similarity |
|---|---|
Layer level (q_proj) | 0.991 |
| Full model (4.41B) | 0.907 |
Hardware Requirements
- Minimum: NVIDIA Blackwell (CC 12.0) or Ada Lovelace for native NVFP4 inference
- Fallback: Dequantizes to BF16 on older hardware
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Created
- Feb 7, 2026
- Updated
- Feb 7, 2026