Downloads · 30 days
0
Meta-Llama-3-8B-KronQ-W2A16
donghyunli/Meta-Llama-3-8B-KronQ-W2A16
Meta-Llama-3-8B-KronQ-W2A16 is a text generation model from donghyunli. Use it when you need the model to write or continue text. The card lists the license as llama3.
Access
Public
Updated Jul 14, 2026
Parameters
3.3B
4.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.9 GB · 100%
Parameter types
How the weights are stored.
U81.7B · 52%
Try a prompt
Base models
Model card
From the Hugging Face model README
Meta-Llama-3-8B — KronQ W2A16 (packed int2)
Paper: arXiv:2607.07964 · Code: GitHub
Meta-Llama-3-8B quantized to 2-bit weights / 16-bit activations with KronQ (Kronecker-factored Hessian quantization). Weights are stored packed int2 (~4.6 GB; the 128k-token embedding + lm_head stay fp16); a fused dequant + bidirectional-incoherence (BiIP) CUDA kernel unpacks them on the fly.
Results (WikiText-2, seqlen 2048)
Perplexity: 11.92
Zero-shot accuracy:
| PIQA | ARC-E | ARC-C | HellaSwag | WinoGrande | BoolQ | OBQA | Average |
|---|---|---|---|---|---|---|---|
| 67.03 | 50.67 | 30.63 | 49.00 | 61.09 | 66.12 | 33.00 | 51.08 |
(lm-evaluation-harness, 0-shot. acc_norm for PIQA/HellaSwag/ARC/OBQA, acc for WinoGrande/BoolQ.)
Usage
KronQ-packed checkpoint (model.safetensors carries biip_w_codes/scale/zero + BiIP buffers, see kronq_packed_config.json). Load with the KronQ runtime:
python eval_pretrained.py meta-llama/Meta-Llama-3-8B donghyunli/Meta-Llama-3-8B-KronQ-W2A16 --ppl --zs
Recipe
Per-channel asymmetric W2, weight-only (a_bits=16), --alpha 0.25, bidirectional incoherence processing (BiIP), act_order, raw H_G. Calibrated on 128 WikiText-2 sequences.
License
Derivative of Meta-Llama-3-8B — subject to the Llama 3 Community License.
- Task
- Text Generation
- License
- llama3
- Languages
- en
- Created
- Jun 22, 2026
- Updated
- Jul 14, 2026