Downloads · 30 days
172
21% of all-time downloads
Llama-3.1-8B-Instruct-FPQuant-QAT-MXFP4
ISTA-DASLab/Llama-3.1-8B-Instruct-FPQuant-QAT-MXFP4
Llama-3.1-8B-Instruct-FPQuant-QAT-MXFP4 is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.1-8B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
All-time downloads
806
Public
Parameters
4.8B
5.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.8 GB · 100%
Parameter types
How the weights are stored.
U83.7B · 78%
Model card
From the Hugging Face model README
This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.1-8B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.
This model can be run on Blackwell-generation NVIDIA GPUs via QuTLASS and FP-Quant in either transformers or vLLM.
The approximate recipe for training this model (up to local batch size and LR) is available here.
This checkpoint has the following performance relative to the original model and the RTN quantization:
| Model | MMLU | GSM8k | Hellaswag | Winogrande | Avg |
|---|---|---|---|---|---|
meta-llama/Llama-3.1-8B-Instruct | 72.8 | 85.1 | 80.0 | 77.9 | 78.9 |
| RTN | 62.6 | 71.8 | 75.2 | 72.3 | 70.5 |
| QAT (THIS) | 67.6 | 80.3 | 78.3 | 74.9 | 75.3 |
- Type
- llama
- Created
- Oct 16, 2025
- Updated
- Oct 27, 2025