taq-qwen32b-coder-outlier-3bit

rshoemake/taq-qwen32b-coder-outlier-3bit

taq-qwen32b-coder-outlier-3bit is a machine learning model from rshoemake. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-Coder-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — uniform outlier action applied to every eligible layer, at a…

Downloads · 30 days

17% of all-time downloads

All-time downloads

Public

Parameters

32.8B

65.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors65.5 GB · 100%

Parameter types

How the weights are stored.

F1631.2B · 95%

Base models

Qwen/Qwen2.5-Coder-32B-Instruct

Model card

From the Hugging Face model README

TAQ Compressed Weights — Qwen2.5-Coder-32B-Instruct, `outlier` (3-bit-class)

Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-Coder-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — uniform outlier action applied to every eligible layer, at a ~3-bit budget.

Fidelity (native family-scale evaluation, WikiText-2)

fp16 reference PPL: 11.9266 (KLD=0 by definition)

	bpw	PPL	KLD vs fp16
This checkpoint (`outlier_3bit`, uniform)	3.8116	13.922	0.1109
`rotation_outlier_3bit` (uniform, same budget class)	3.8259	13.793	0.1307
`mixed_allocator_3bit` (per-layer allocator, same budget class)	3.8116	13.194	0.0884

outlier beat rotation_outlier on fidelity at this budget (0.1109 vs 0.1307 KLD).

Code generation quality (EvalPlus HumanEval+ / MBPP+, pass@1)

	HumanEval Base	HumanEval Base+Extra	MBPP Base	MBPP Base+Extra
This checkpoint (`outlier_3bit`, uniform)	0.8780	0.8354	0.8571	0.7368
`rotation_outlier_3bit` (uniform, same budget class)	0.9085	0.8354	0.8496	0.7168
`mixed_allocator_3bit` (per-layer allocator, same budget class)	0.9024	0.8476	0.8471	0.7243
fp16 (reference, uncompressed)	0.9024	0.8598	0.8647	0.7268
Unsloth bnb-4bit (vendor comparator, ~6.1 bpw)	0.9024	0.8720	0.8596	0.7293
GGUF Q4_K_M (Unsloth, vendor comparator, ~4.85 bpw)	0.8902	0.8293	0.8546	0.7293

vs fp16: HumanEval Base+Extra -0.0244, MBPP Base+Extra +0.0100 (pass@1 delta, this checkpoint minus fp16 baseline).

What's in this repo

Real, byte-packed quantized weights (not a dequantized fp16 scaffold) — same format documented in full at rshoemake/taq-qwen14b-unsloth-matched: packed/*.bin per-layer files (header + fp16 codebooks + bit-packed indices + outlier side-channel), packed/manifest.json (per-layer action/bits/measured bpw), fp16_passthrough/*.safetensors for any layers the allocator left at full precision, unpack_outlier.py / unpack_rotation_outlier.py standalone unpackers.

Provenance

Source: Qwen/Qwen2.5-Coder-32B-Instruct (fp16), packed directly from the original safetensors shards.
Fidelity evaluation harness: run_full_matrix.py / run_tail_family_scale.py (project fractal_quant_decision, private) — native family-scale protocol (WikiText-2, passage-level KLD).
Code-eval harness: code_eval_gen.py + evalplus.evaluate (HumanEval+/MBPP+, greedy decoding, max-new-tokens=640) — same decode/scoring config used across every arm in this comparison table so results are directly comparable.
Packer: pack_outlier_format.py + pack_rotation_outlier.py + pack_whole_model.py.

Type: qwen2
License: apache-2.0
Created: Aug 11, 2026
Updated: Aug 15, 2026