Downloads · 30 days
86
41% of all-time downloads
Llama-3.2-1B-Instruct-FPQuant-QAT-NVFP4
ISTA-DASLab/Llama-3.2-1B-Instruct-FPQuant-QAT-NVFP4
Llama-3.2-1B-Instruct-FPQuant-QAT-NVFP4 is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.2-1B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
All-time downloads
210
Public
Parameters
810M
1.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 98%
Parameter types
How the weights are stored.
U8547M · 68%
Model card
From the Hugging Face model README
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.2-1B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
This model can be run on Blackwell-generation NVIDIA GPUs via QuTLASS and FP-Quant in either transformers or vLLM.
The approximate recipe for training this model (up to local batch size and LR) is available here.
This checkpoint has the following performance relative to the original model and the RTN quantization:
| Model | MMLU | GSM8k | Hellaswag | Winogrande | Avg |
|---|---|---|---|---|---|
meta-llama/Llama-3.2-1B-Instruct | 46.2 | 46.3 | 59.8 | 61.6 | 53.5 |
| RTN | 32.8 | 25.0 | 56.2 | 59.0 | 43.3 |
| QAT (THIS) | 32.7 | 37.6 | 57.5 | 58.4 | 46.6 |
- Type
- llama
- Created
- Oct 16, 2025
- Updated
- Oct 27, 2025