GLM-5.3-Flash-NVFP4-attn-lmhead-W4A16

NVFP4 BIZ AXL: GLM-5.3-Flash-NVFP4, attention + lm_head in W4A16

Short name: NVFP4 BIZ AXL (AXL: the attention projections and lm_head in W4A16). It is the published option of the serving stack GLM-5.3-Flash BIZ (Bizuayeu/GLM-5.3-Flash-DGX-Sparks-BIZ) in its 1.x line (vLLM) and, from 2.4.0, its 2.x line (TensorFold); this repository's name describes the contents and stays as it is. Cite a revision as "NVFP4 BIZ AXL (<revision>)".

A derived checkpoint of nvidia/GLM-5.3-Flash-NVFP4. The attention-side linears and lm_head, which NVIDIA leaves in BF16, are repacked weight-only to W4A16 NVFP4; every other tensor is byte-identical to the source. It is not an NVIDIA or Z.AI release.

Use it for generation-heavy serial work, Japanese prose in particular, on GB10 (DGX Spark class) systems served by GLM-5.3-Flash BIZ 1.x (vLLM), which serves two hosts at TP=2: decode is about 1.4 times the source checkpoint's at the same draft depth, prefill is 2% faster, and about 4.4 GiB less is loaded per rank. That saving is what pays for a second sequence: the setup's AXL example profile serves two active sequences from 6 GiB of KV per rank, accepted there for routine use since 2026-09-23 at up to about 200K tokens each. Do not use it where the pinned weights must be matched exactly (code, tool use): it is not lossless — teacher-forced NLL is 1 to 6% higher on three of 1.x's four test corpora, and higher on all four of 2.x's NLL set.

GLM-5.3-Flash BIZ 2.x (TensorFold) serves these weights too, from 2.4.0, as an option beside the source checkpoint, one sequence at a time: its engine reads the W4A16 attention and head by their tensors, without the overlays, and every rank file's CHECKPOINT chooses them (loading). What 2.x measured on them is in its measured on the release. The overlays below belong to the 1.x line in the setup repository's v1/. The 3.x line (TensorFold's native engine) has not accepted these weights.

At a glance

Requirements

This section is about vLLM, the 1.x line's engine; the 2.x line needs none of it. The pinned vLLM (385dce36) does not route these modules through its NVFP4 path on its own, and this checkpoint declares the KDA input projection as q_proj / k_proj / v_proj plus one merged in_proj_bfg_a (b, f_a, g_a) instead of vLLM's six-in-one in_proj_qkvbfg_a. Loading therefore needs the two source overlays that the setup repository ships in its overlays/ directory:

An overlay pair belongs to one revision of this checkpoint (see Revisions); a mismatched pair fails at load. The KDA overlay above is the setup's 1.24.0 version: it copies f_a and g_a into fresh storage before their Marlin projections, which TP=3 (22 heads per rank) needs. At TP=2 it gave the same decode-check hashes and teacher-forced log-probabilities, bit for bit, as the earlier 144a835f…, which TP=2 profiles may keep. Plain vllm serve on this folder without the overlays does not start.

Above about 250K tokens this checkpoint also needs the slot-mapping guard (patch_slot_mapping, vLLM issue #53982) that the setup's images carry from 1.7.0: without it, the pinned vLLM's slot-mapping kernel reads past the end of a block-table row, and with this checkpoint's memory layout that faulted on both ranks between 248,954 and 252,958 tokens (details). On a runtime without the guard, keep requests at or below 245,000 tokens.

Loading

• With the setup's 1.x line (vLLM, recommended): set runtime.derived_checkpoint in the server TOML — path to this folder, requant_target = "l", and the two overlays with their SHA-256 and base SHA-256 — and start through its launcher. server preflight refuses the launch unless each overlay has its declared hash and marker and the image's own target file has the declared base hash, so an overlay made for another image cannot be mounted. examples/server.axl.example.toml carries all of it, with prefix-page dedup and two active sequences; replace its paths, image IDs and node details, and set each host's optional cpuset_cpus to performance cores from that host's own topology (CPU placement). Details: server configuration.
• On vLLM without the setup: copy the two overlays over the two files named above inside a vLLM 385dce36 installation, then vllm serve this folder; the declarations in config.json then reach the declared modules. Mind the 250K limit above.
• With the setup's 2.x line (TensorFold): download them as its setup does and set CHECKPOINT=/hub/models--Bizuayeu--GLM-5.3-Flash-NVFP4-attn-lmhead-W4A16/snapshots/bbad98c8f380588c16a2326bf5a2ab7344190b07 in every rank file (configuration); no overlay.

Measured against the source checkpoint

Two GB10, TP=2, MTP depth 3, temperature 0, the setup's 1.19.0 image. Both columns were measured on 2026-09-28 in one window: the pair switched from this checkpoint's serving profile to the source checkpoint, then to this checkpoint at one sequence (for repeatability), then back. This checkpoint is the profile the setup's reference pair serves (the AXL example: prefix-page dedup, two active sequences, 6 GiB of KV per rank), measured with one request in flight unless the row says otherwise. The source checkpoint is the setup's distributed defaults (one sequence, 3 GiB of KV per rank). Both ranks' workers ran on performance cores (CPUs 5–9 and 15–19); with either rank on efficiency cores decode falls to about a third. All nodes ran with the GPU clock capped at 2,200 MHz, and each stage waited for temperatures to come down first; against this checkpoint's profile without the cap (2026-09-27), prefill was 1.8% slower and long inputs 0.8–1.9% slower. Medians of three runs; ranges and conditions are in the setup repository's benchmarks.

The sparkDash code prompt changed from 108 to 66 tokens on 2026-09-26, so its code column cannot be compared with this card's earlier revision.

Quality, teacher-forced NLL on the setup's four fixed corpora (same inputs; the serving profile, two sequences, deterministic Inductor configs; the same values on the setup's 1.12, 1.13 and 1.19 images):

The earlier figures (2026-09-21, one-sequence profile before the setup's 1.12 launch-state fix) were 1.6645 / 2.0024 / 1.0031 / 0.6279.

Since the setup's 1.12.0 (deterministic Inductor configs on both ranks), every launch checked has computed in the same numerical state; before it, relaunches fell into one of three states that differed in how a near tie falls. The setup's 1.19.0 ports vLLM #58454 (the kpool tail ring), which can change the compressed keys of pools built during decode past the indexer's index_topk (2,048): reference completions recorded on earlier images are not a baseline for a context past that length. The setup's 1.x line checks every launch after a switch (weight digest, decode check, kernel hashes on both ranks); see its launch checks. Decode speed under MTP depends on how predictable the text is (counting is about 1.5 times as fast as prose); the draft is the checkpoint's own MTP layer, no extra weights.

Revisions

The weights and declarations have not changed since 2026-09-22; later commits edit this card only.

License and notices

The weights are redistributed under the MIT terms stated by the NVIDIA model card and by the upstream Z.AI license; both notices are preserved (README.nvidia.md, and the Z.AI license text in the setup repository's LICENSES/). This repackaging adds no training data and no new terms. The conversion tool is Apache-2.0 and its license applies to the tool, not to these weights. No warranty; see the source model cards for intended use and limitations.