GLM-5.2-SQG-W4A8

GLM-5.2 SQG W4A8

This is an all-SQG, BF16-source-only GLM-5.2 checkpoint. Routed expert tensors use independent per-tensor K3/K4 assignments (384 of each per layer); the 380 selected non-routed matrices use SQG K6. It contains 58,748 SQG matrices and zero MCG marker tensors. Routed experts use the full-W4A8 endpoint: both h and SiLU(gate) * up are A8. Dense K6 matrices are calibrated in that construction context but materialized and served as native SQG W6A16.

Routed gate/up calibration uses the frozen BMM Law blend H13 = 0.75 * H13_layer + 0.25 * H13_expert, with routed applied-router-gate-square weighting and the sealed per-tensor rate map. Down projections are encoded from the candidate-conditioned W4A8 path using the recorded (H, B) cross-term construction. Hadamard rotations and input-side scales remain on the activation side so native SQG E4M3 weight labels reach FP8 MMA unchanged.

Release state: owner-directed unvalidated release on main; KLD and runtime acceptance were skipped.

Release regime

• Tensor parallel: 1
• Pipeline parallel: 8, partition [9, 10, 10, 10, 10, 10, 10, 9]
• Final-logit KLD: not run (owner-directed speed publication)
• Runtime acceptance: not run (owner-directed speed publication)
• Official BF16 revision: b4734de4facf877f85769a911abafc5283eab3d9
• Calibration/Hessian dataset: db55cc25ea16c49b458d1a3f1a4661095be96ba4
• Canonical deduplicated calibration view: canonical-v1

This format requires the custom SQG W4A8 vLLM runtime. It is not a drop-in Transformers checkpoint. Do not start it as TP8, DCP4, an SM120-only MCG image, or an A16 fallback. The accepted topology is PP8/TP1 and the supplied launch artifacts enforce it.

Run

Set VERDICTAI_IMAGE to the immutable image tested for this release:

export VERDICTAI_IMAGE=verdictai/glm52-sqg-w4a8-sm103@sha256:53641269fccf60a80e2467cc8502dc273fe4d0bfb33bbf2bd894d36262fe08f2
./serve.sh start
./serve.sh logs

The API binds to 127.0.0.1:8000 by default. Exact construction, runtime, acceptance, and dataset identities are recorded in RELEASE_PROVENANCE.json and the two FULL_SQG_NATIVE_*.json files.

Reproduce the quantization

See REPRODUCE_SQG_W4A8.md for the complete BF16-source capture, H13/H2/B calibration, per-tensor K3/K4 routed encoding, dense-K6 encoding, materialization, and publication procedure. Exact sealed scripts and source hashes are in the linked calibration dataset under reproduction/ and derived/lineage/reproducibility/.

HESSIAN_DATASET_LAYOUT.md identifies the ready-to-use H13 files, canonical activation views, dense-H inputs, and intentionally repeated archival paths so users do not download all 3.1 TB unnecessarily.

Credits and lineage

• Z.ai for GLM-5.2.
• Luke Alonso for QSRT/KQuant and B12X work. The QSRT/KQuant source history from initial commit 3acea57fb5a82b3b848504bdc87953420f531f39 through audited snapshot 104dd9233f850a3955f4991bea68b07dd34deeb8 is authored by Luke. The repository is not linked because no working public URL was verified.
• Brandon Music for the GLM-specific corpus, Hessians, encoder adaptations, model assembly, validation, and publication.
• turboderp for the reused ExLlamaV3 v0.0.43 encoder, trellis-reference, compatibility, and codebook components present in the published reproduction source. EXL3's documented research lineage includes QTIP and QuIP#; those papers are not authors of the custom SQG method.
• Special thanks to local-inference-lab for its vLLM fork, Blackwell runtime/image work, testing, and review.

The exact source and calibration artifacts are the linked, revision-pinned Hessian datasets above.

License

This repository does not relicense GLM-5.2 or third-party runtime code. The derived checkpoint remains subject to upstream GLM-5.2 terms; see LICENSE. Reused ExLlamaV3 components remain subject to its MIT license. The audited QSRT/KQuant snapshot contains no top-level license declaration, so this card does not infer one; resolve that source-license notice before redistributing those source files independently.