rebuttal_w4pc_cbq

Qwen3-1.7B — CBQ W4-per-channel

Qwen/Qwen3-1.7B quantized to W4-per-channel with CBQ (Cross-Block Quantization, arXiv:2312.07950), then dequantized back to bf16.

This is a dense bf16 checkpoint, not a packed low-bit one. Every weight has been round-tripped through the W4-per-channel grid, so it carries the accuracy of W4-per-channel quantization, but the file itself is an ordinary Qwen3ForCausalLM in bfloat16 with the same tensor names and shapes as the base model. It loads in vLLM or transformers with no custom kernel, no quantization_config, and no code change — and it gives no speed or memory benefit over the base model. Use it to evaluate what W4-per-channel costs in quality.

from vllm import LLM
llm = LLM("ghostchoir/rebuttal_w4pc_cbq")            # dtype is bfloat16 per config.json

Configuration

Calibration: 4096 rows of open-r1/OpenR1-Math-220k rendered with the Qwen3 chat template and truncated to 2048 tokens. This is not the paper's calibration set (the paper uses 128 samples); it is matched to a sibling QAT framework so the two are comparable.

Perplexity above is on held-out rows of the same distribution and is a sanity metric only — it is not a reasoning-benchmark score.

Provenance and caveats

Produced by an unofficial CBQ implementation written from the paper text; CBQ has no official code release. The paper is ambiguous or self-inconsistent in several places (notably the interaction of Eq. 8 and Eq. 11 for Delta_W initialisation, and the direction of the Algorithm 1 objective), and every such choice is documented with the measurement behind it. Two settings differ from the paper deliberately: group-wise rather than per-channel quantization where noted, and the calibration set above.

Not affiliated with the CBQ authors or with Qwen. Inherits the base model's license.