Downloads · 30 days
3
10% of all-time downloads
taq-qwen32b-outlier-3bit
rshoemake/taq-qwen32b-outlier-3bit
taq-qwen32b-outlier-3bit is a machine learning model from rshoemake. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — uniform outlier action applied to every eligible layer, at a ~3-bi…
All-time downloads
30
Public
Parameters
32.8B
65.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors65.5 GB · 100%
Parameter types
How the weights are stored.
F1631.2B · 95%
Base models
Model card
From the Hugging Face model README
TAQ Compressed Weights — Qwen2.5-32B-Instruct, outlier (3-bit-class)
Real, byte-packed, kernel-loadable quantized weights for
Qwen/Qwen2.5-32B-Instruct, produced by the project's Tail-Aware
Quantization (TAQ) pipeline — uniform outlier action applied to every eligible layer, at a ~3-bit budget.
Fidelity (native family-scale evaluation, WikiText-2)
fp16 reference PPL: 5.4541 (KLD=0 by definition)
| bpw | PPL | KLD vs fp16 | |
|---|---|---|---|
This checkpoint (outlier_3bit, uniform) | 3.8116 | 8.176 | 0.5431 |
rotation_outlier_3bit (uniform, same budget class) | 3.8259 | 8.144 | 0.5681 |
mixed_allocator_3bit (per-layer allocator, same budget class) | 3.8116 | 7.506 | 0.4631 |
outlier beat rotation_outlier at this budget (0.5431 vs 0.5681 KLD).
What's in this repo
Real, byte-packed quantized weights (not a dequantized fp16 scaffold) —
same format documented in full at
rshoemake/taq-qwen14b-unsloth-matched:
packed/*.bin per-layer files (header + fp16 codebooks + bit-packed
indices + outlier side-channel), packed/manifest.json (per-layer
action/bits/measured bpw), fp16_passthrough/*.safetensors for any
layers the allocator left at full precision, unpack_outlier.py /
unpack_rotation_outlier.py standalone unpackers.
Provenance
- Source:
Qwen/Qwen2.5-32B-Instruct(fp16), packed directly from the original safetensors shards. - Evaluation harness:
run_full_matrix.py/run_tail_family_scale.py(projectfractal_quant_decision, private) — native family-scale protocol (WikiText-2, passage-level KLD). - Packer:
pack_outlier_format.py+pack_rotation_outlier.py+pack_whole_model.py. - No vendor-matched (Unsloth bpw-matched) comparison has been run at this scale yet — this card reports fidelity against the fp16 baseline and against this project's own uniform baselines only.
- Type
- qwen2
- License
- apache-2.0
- Created
- Aug 11, 2026
- Updated
- Aug 14, 2026