Downloads · 30 days
56
13% of all-time downloads
Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4
Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4 is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.2-3B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
All-time downloads
426
Public
Parameters
2B
2.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors2.4 GB · 99%
Parameter types
How the weights are stored.
U81.6B · 80%
Model card
From the Hugging Face model README
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.2-3B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
This model can be run on Blackwell-generation NVIDIA GPUs via QuTLASS and FP-Quant in either transformers or vLLM.
The approximate recipe for training this model (up to local batch size and LR) is available here.
This checkpoint has the following performance relative to the original model and the RTN quantization:
| Model | MMLU | GSM8k | Hellaswag | Winogrande | Avg |
|---|---|---|---|---|---|
meta-llama/Llama-3.2-3B-Instruct | 64.4 | 78.0 | 73.4 | 70.1 | 71.5 |
| RTN | 59.9 | 64.8 | 69.8 | 65.6 | 65.0 |
| QAT (THIS) | 62.0 | 72.9 | 71.0 | 66.5 | 68.1 |
- Type
- llama
- Created
- Oct 16, 2025
- Updated
- Oct 27, 2025