GLM-5.3-EXL3-4.0bpw

[!WARNING] Known KL regression — do not use this checkpoint yet. Measured KL vs BF16 is ~1.02 nats (reproduced: 1.0223 then 1.0215; median token KL 0.33), ~10× worse than expected for an experts-only + fp16-sensitive build. The regression is confirmed real (verified-complete checkpoint, measured twice), and traced to a crash-and-resume finalization during the encode. A clean full re-encode is pending; until then use the 3bpw ladder or the W4A16 cuts.

[!TIP] Support this work → · X · GitHub · REAP paper · Cerebras REAP

GLM-5.3 — EXL3 4.0bpw (experts-only quant, BF16 sensitive layers)

The full GLM-5.3 (all 256 experts) where only the routed experts are quantized to 4-bit and every sensitive layer stays full-precision BF16 — 405 GB. This is the high-fidelity build and the unpruned reference for the 4bpw series.

What this is

GLM-5.3 is a 753B mixture-of-experts model (8 of 256 experts per token, ~40B active). Here only the routed experts are EXL3 4bpw (mcg codebook); attention (including the DSA indexer), the shared expert, the dense layers, and the head are kept BF16 byte-exact — the davidsyoung/brandonmusic standard. Compression is spent only where the model is most redundant, so this sits far closer to the original than a fully-quantized 3-bit checkpoint.

Pruned, VRAM-friendlier cuts of this model are in the EXL3 collection. Criterion and all KL runs: fidelity study.

Running it

pip install exllamav3   # or via TabbyAPI; tensor-split across your GPUs

Credits

• Z.AI / zai-org — GLM-5.3, the base model.
• Cerebras Research — REAP (arXiv:2510.13999).
• turboderp / exllamav3 — the EXL3 format and runtime.

Observations: glm-5.3-reap-observations-v1 · Built on 8× NVIDIA RTX PRO 6000 Blackwell.

License

Inherits the GLM-5.3 license.