Downloads · 30 days
42
22% of all-time downloads
Qwen3-8B-FPQuant-QAT-NVFP4
ISTA-DASLab/Qwen3-8B-FPQuant-QAT-NVFP4
Qwen3-8B-FPQuant-QAT-NVFP4 is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is the official QAT FP-Quant checkpoint of qwen/Qwen3-8B, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
All-time downloads
194
Public
Parameters
5.2B
6.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors6.4 GB · 100%
Parameter types
How the weights are stored.
U83.9B · 76%
Model card
From the Hugging Face model README
This is the official QAT FP-Quant checkpoint of qwen/Qwen3-8B, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
This model can be run on Blackwell-generation NVIDIA GPUs via QuTLASS and FP-Quant in either transformers or vLLM.
The approximate recipe for training this model (up to local batch size and LR) is available here.
This checkpoint has the following performance relative to the original model and the RTN quantization:
| Model | MMLU | GSM8k | Hellaswag | Winogrande | Avg |
|---|---|---|---|---|---|
qwen/Qwen3-8B | 73.0 | 90.9 | 75.5 | 70.6 | 77.5 |
| RTN | 70.2 | 86.4 | 73.0 | 68.1 | 74.4 |
| QAT (THIS) | 71.3 | 89.2 | 75.2 | 70.4 | 76.5 |
- Type
- qwen3
- Created
- Oct 16, 2025
- Updated
- Oct 27, 2025