Downloads · 30 days
0
donghyunli/Llama-2-13b-KronQ-W4A16
Llama-2-13b-KronQ-W4A16 is a text generation model from donghyunli. Use it when you need the model to write or continue text. The card lists the license as llama2.
Downloads · 30 days
0
Access
Public
Updated Jul 14, 2026
Parameters
6.8B
7.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7.4 GB · 100%
How the weights are stored.
U86.3B · 93%
From the Hugging Face model README
Paper: arXiv:2607.07964 · Code: GitHub
Llama-2-13b quantized to 4-bit weights / 16-bit activations with KronQ (Kronecker-factored Hessian quantization). Weights are stored packed int4 (~6.9 GB vs 26 GB fp16); a fused dequant + bidirectional-incoherence (BiIP) CUDA kernel unpacks them on the fly.
Perplexity: 4.96
Zero-shot accuracy:
| PIQA | ARC-E | ARC-C | HellaSwag | WinoGrande | BoolQ | OBQA | Average |
|---|---|---|---|---|---|---|---|
| 80.20 | 74.71 | 47.95 | 79.14 | 71.82 | 82.14 | 44.80 | 68.68 |
(lm-evaluation-harness, 0-shot. acc_norm for PIQA/HellaSwag/ARC/OBQA, acc for WinoGrande/BoolQ.)
KronQ-packed checkpoint (model.safetensors carries biip_w_codes/scale/zero + BiIP buffers, see kronq_packed_config.json). Load with the KronQ runtime:
python eval_pretrained.py meta-llama/Llama-2-13b-hf donghyunli/Llama-2-13b-KronQ-W4A16 --ppl --zs
Per-channel asymmetric W4, weight-only (a_bits=16), --alpha 0.25, bidirectional incoherence processing (BiIP), act_order, raw H_G. Calibrated on 128 WikiText-2 sequences.
Derivative of Llama-2-13b — subject to the Llama 2 Community License.