circus-0.4-t9

circus-0.4-t9 — 9-level additive bit-plane QAT recovery of Agents-A1-4B

Author: Wei-Ciao Wu (@wcamon) Code: github.com/wcAmon/guava-qat — full quantization / reconstruction / eval facility that produced this model. Status: stage result (v0.4.1, "t9" = 9-level grid) — ships both the bf16 materialization and the true packed bit-planes (bitplanes_k2_c0.6.npz, bit-exact, see below); addition-only kernels to follow.

Every linear weight of Agents-A1-4B is constrained to the additive two-plane 9-level grid

W = α ⊙ (T₁ + c·T₂),  T₁,T₂ ∈ {−1,0,+1},  c = 0.6,  α: one scale / 32 weights
grid = {0, ±0.4, ±0.6, ±1, ±1.6}   (≈ 3.67 bits/weight, ≈ 4.4× vs bf16)

The two ternary planes admit addition-only matmul kernels (BitNet-style), and 7/5-level variants of the same family cost the same at inference — 9-level is the capability ceiling of the family, single-plane ternary its compression ceiling.

Two artifacts, one model

1. model.safetensors — the quantized values materialized in bf16, so the model loads with stock transformers exactly like the base model. This is a storage/compatibility format, not a claim of 16-bit information content: every weight lies on its block's 9-level grid.

2. bitplanes_k2_c0.6.npz — the true quantized artifact (1.7 GB vs 8.5 GB): per target linear, 4-bit plane indices idx=(T₁+1)·3+(T₂+1), two per byte, in the GPTQ column-permuted domain, plus per-block-32 fp32 scales α and the int32 inverse column permutation inv. Scales are kept fp32 so that decoding is bit-exact against model.safetensors:

   python decode_bitplanes.py     # verified: 200/200 tensors bit-exact
   

   Raw pack ≈ 5 bits/weight (4-bit index + fp32 scale); with fp16 scales and entropy coding of the 9-way index the format reaches ≈ 3.67 bits/weight. No addition-only matmul kernel ships yet — until one exists, the bf16 materialization is how you run the model, the bit-plane pack is what the model is.

Non-target weights (embeddings, lm_head, layernorms, the vision stack) remain bf16 in both forms.

Recovery recipe (single MI325X, < 5 GPU-hours total)

1. GPTQ-style per-layer init with jointly solved block scales.
2. Sliding-window soft-anneal reconstruction (3 h): windows of 4 layers, stride 2, per-window multi-threshold tanh softening annealed s→30 over 80 % of epochs, explicit STE hard finish for the last 20 %, LoRA r=64 + multiplicative scale modulation as carriers, huber loss against the fp window's own outputs on 512 calibration segments. This puts the ternary assignment T into the gradient loop (22 % of assignments flip) — the step that end-to-end scale-only KD provably could not do in our ablations. Follows the softened-ternarization + sliding-layer reconstruction line of CAT-Q (ICML'26) / SliderQuant (ICLR'26), generalized from ternary to the additive 9-level grid.
3. Light polish (9 min): 50 steps of scale-only logit KD from the bf16 teacher.

Results (retention vs. bf16 base, same local harness, full test sets)

All ten tracked tasks retain ≥ 90 % (mbpp excluded from the headline because its train split occurs in the polish corpus). Scores are lm-eval-harness, no chat template, greedy/likelihood defaults; single seed; your numbers may differ under other harnesses.

Honest scope

• At ~3.67 bits this is an easier target than pure ternary (CAT-Q) — the point of the release is the additive bit-plane form (addition-only kernels, 9/7/5 same-cost family) and the recovery methodology, not a bit-budget record.
• Findings we believe transfer: (i) at this bit budget, the recovery bottleneck is assignment quality, not scales — local fp-anchored reconstruction fixes in hours what end-to-end KD cannot fix at all; (ii) val CE is not a valid intermediate judge for low-bit recovery (our best model is 0.42 nats worse in CE than a strictly weaker checkpoint); (iii) corpus-mix Pareto trade-offs observed during scale-only polishing dissolve once assignments are repaired.
• Base model is a VLM; only the language stack was quantized and evaluated.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("wcamon/circus-0.4-t9", torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("wcamon/circus-0.4-t9")

Citations

CAT-Q (arXiv:2606.26650) · SliderQuant (ICLR 2026) · BitNet b1.58 (arXiv:2402.17764) · ParetoQ (arXiv:2502.02631) · GPTQ (arXiv:2210.17323) · BRECQ (arXiv:2102.05426) · STE (arXiv:1308.3432) · Agents-A1 (arXiv:2606.30616)