Downloads · 30 days
23
25% of all-time downloads
Qwen3-4B-FP8
vistralis/Qwen3-4B-FP8
Qwen3-4B-FP8 is a text generation model from vistralis. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
FP8 (W8A8) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
All-time downloads
91
Public
Parameters
4.4B
5.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.2 GB · 100%
Parameter types
How the weights are stored.
F8_E4M33.6B · 82%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-4B-FP8
FP8 (W8A8) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
Overview
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-4B |
| Parameters | 4.41B |
| Quantization | FP8 (W8A8) |
| Format | compressed-tensors |
| Tool | llm-compressor |
| Disk Size | ~4.9 GB (2 shards) |
| VRAM | ~3.96 GB |
Intended Use
Quantized text encoder for Flux 2 Klein 4B image generation pipelines. Architecturally identical to the Klein 4B text encoder.
Quantization Details
- Scheme: FP8 — 8-bit float weights and activations (
float8_e4m3fn) - Targets: All
Linearlayers (excludinglm_head) - Calibration: 256 samples, sequential pipeline with CPU offloading
Hardware Requirements
- Minimum: NVIDIA Hopper (CC 8.9+) or Ada Lovelace for native FP8 inference
- Fallback: Dequantizes to BF16 on older hardware
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Created
- Feb 7, 2026
- Updated
- Feb 7, 2026