Downloads · 30 days
26
22% of all-time downloads
Qwen3-4B-INT8
vistralis/Qwen3-4B-INT8
Qwen3-4B-INT8 is a text generation model from vistralis. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
INT8 (W8A8) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
All-time downloads
119
Public
Parameters
4.4B
5.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.2 GB · 100%
Parameter types
How the weights are stored.
I83.6B · 82%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-4B-INT8
INT8 (W8A8) quantized version of Qwen/Qwen3-4B, created using llm-compressor with calibrated quantization.
Overview
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-4B |
| Parameters | 4.41B |
| Quantization | INT8 (W8A8) |
| Format | compressed-tensors |
| Tool | llm-compressor |
| Disk Size | ~5.2 GB (2 shards) |
Intended Use
Quantized text encoder for Flux 2 Klein 4B image generation pipelines. Architecturally identical to the Klein 4B text encoder.
Quantization Details
- Scheme: W8A8 — 8-bit integer weights and activations
- Targets: All
Linearlayers (excludinglm_head) - Calibration: 256 samples from C4, sequential pipeline with CPU offloading
Hardware Requirements
- Minimum: Any CUDA GPU with INT8 tensor core support
- Fallback: Dequantizes to BF16 on unsupported hardware
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Created
- Feb 7, 2026
- Updated
- Feb 7, 2026