GLM-5.3-Flash-EXL3-Spark

GLM-5.3-Flash EXL3 — Spark

A mixed-precision EXL3 build of GLM-5.3-Flash sized to serve a 256k-token context with vision on a single NVIDIA DGX Spark (GB10, arm64, 128 GB unified memory): 12 of the 45 routed-expert MoE layers are quantized at 3.05 bpw for accuracy, the other 33 stay at 2.05 bpw for size. End result: ≈2.31 bpw average, 96.1 GB on disk, and higher agreement with the teacher than the 2.05 bpw base it is built from.

This is a quality artifact, not a speed one. It currently runs without speculative decoding (see MTP), which roughly halves decode rate against EXL3 builds that do. Read that section before deploying.

Previously published as GLM-5.3-Flash-EXL3-M288-Mosaic-12L; the weights are unchanged.

At a glance

What "≈2.31 bpw" means here

bpw is the trellis rate of the quantized weights. This artifact has exactly two rates, chosen per layer:

The average is 2.31 bpw, and it can be derived two independent ways that agree:

• By bytes. The artifact is 12.76 % larger than its 2.05 bpw base (96.105 GB vs 85.233 GB), and the only change is the rate of those 12 layers, so 2.05 × 96.105/85.233 = 2.3115.
• By layer mass. Upgrading a layer from 2.05 to 3.05 bpw multiplies its expert bytes by 1.488. The measured delta (10.87 GB) therefore implies those 12 layers hold 22.29 GB in the base — 26.15 % of the file for 26.67 % of the layers, i.e. the layers are uniform. The equal-layer average is (33×2.05 + 12×3.05)/45 = 2.3167.

Both land on ≈2.31 bpw. The dense/attention dressing, embeddings and vision tower are unchanged from the base and are not part of the rate change; they are what pulls the two derivations apart in the third decimal.

Why not uniform 2.31 bpw everywhere? Because the 12 chosen layers are the ones where extra precision buys the most agreement, and spending it there beats spending it evenly. The panel below is the measurement of that bet.

Measured quality

Full G4 panel, 65,504 positions across 32 pre-registered rows, same harness, same image and same teacher rows on both sides; teacher reproduction exact on both (top1_exact_all_rows: true). The 2.05 bpw side was re-measured the same day rather than quoted.

32/32 rows improve on top-1, 32/32 on KL, 32/32 on NLL. No row regresses.

Task benchmarks:

Within the program that produced it, this is the only artifact that beats the 2.05 bpw base on the panel and regresses on neither benchmark. Two earlier mosaics (e216, f216) sit 4–6 pp below the base on both.

Serving on one DGX Spark

Validated 2026-09-18 on a Spark from a directory downloaded through the published path, so these are user-facing numbers:

Measured with a streaming sweep whose every prompt opens with a unique nonce, so prefix caching cannot flatter the result; the prefill fit's small residuals are themselves the evidence that each prompt paid a full cold prefill. Server: SGLang EXL3, fp8 KV, DSA attention, mem-fraction-static 0.95 (KV pool 595,200 tokens), max-running-requests 1, no CUDA graphs.

For scale: EXL3 builds that run native MTP on the same box reach 18.7–19.6 tok/s decode.

Vision

The image processor declares an 8,000-token ceiling per image, and the 4096×4096 probe lands at 7,921 — i.e. full resolution up to the artifact's own limit. The model reads the fixture's geometry correctly at 512² and 2048² (it reports bar columns at x≈70–135 and x≈290–525, matching the encoder's 73/292 column origins). A verified end-to-end answer to the probe's decoding task was not obtained because the task does not fit its token budget; that limit is on the probe, not the vision path.

MTP: two code blockers, not a weight problem

The MTP layer is present in these weights. Layer 45 is the multi-token-prediction block — eh_proj, enorm, hnorm, shared_head alongside its attention and MLP, with all 288 experts, 873 mul1 tensor sets and 12 native tensors — and config.json declares num_nextn_predict_layers: 1.

What is missing is runtime support. Two code blockers stand in the way, and an experiment on 2026-09-18 isolated the second:

1. The codebook gate. MTP-capable EXL3 runtimes in this lineage implement only the mcg codebook and reject this artifact's mul1 at config validation, before any weight is read: ValueError: this overlay only implements codebook=mcg; got 'mul1'.
2. A checkpoint-key mismatch. With that gate relaxed, validation passes and the engine dies at weight loading — at the first MoE layer, before any kernel runs:

File ".../vllm/models/glm5next/nvidia/model.py", line 904, in load_weights
    param = params_dict[name]
KeyError: 'layers.0.mlp.down_proj.mul1'

These weights descend from the turboderp/SGLang line and carry model.language_model.layers.N.mlp.experts.E.* keys, while the vLLM MTP loader addresses experts as layers.N.mlp.down_proj.*. Layer 0 is a stock, non-upgraded layer, so this is a naming-lineage difference rather than anything caused by the 12 upgraded layers.

Bringing MTP to this artifact therefore needs a checkpoint-key remap first, then a mul1 decode path — code work with a known entry point (model.py:904), not a research problem. Until then the artifact runs without speculative decoding, which is the whole of the decode gap above.

Files

Usage

EXL3 weights need an EXL3 runtime — plain transformers will not execute them. The recipe repository has the launcher, a census preflight that refuses to start on a layout mismatch, and the exact flag set:

• recipe + launcher: https://github.com/0xSero/glm-5.3-flash-spark-mosaic
• sibling 1× Spark EXL3 + MTP work: https://github.com/0xSero/GLM-5.3-Flash-EXL3-2x-DGX-Sparks
• full validation pass (scans, sweeps, cache behaviour, MTP isolation): that fork's docs/VALIDATION.md

Download and verify:

git clone https://github.com/0xSero/glm-5.3-flash-spark-mosaic
cd glm-5.3-flash-spark-mosaic/docker
./download-mosaic.sh ~/models/glm53-exl3-spark     # pinned revision + manifest verification
GLM53_MEM_FRACTION=0.95 GLM53_MAX_RUNNING=1 ./run-spark.sh ~/models/glm53-exl3-spark

Verification

Per-tensor: every upgraded expert tensor is byte-equal to its 3.05 bpw source and every kept tensor is byte-equal to the 2.05 bpw base; the checks are per-tensor and recorded in PROVENANCE.json. The packer's whole-file delta check reports a FAIL on this artifact because it compares total bytes against a prediction and trips on +10,706 B of safetensors header drift across the five rewritten shards — a false positive; the per-tensor checks are the load-bearing ones, and the 10-layer sibling shows the identical drift.

Whole-file: sha256-manifest.txt covers all 26 content files. Verified end to end on 2026-09-18 — 27/27 files fetched through the documented path, sha256sum -c 26/26 OK, exit 0, and the bytes actually served by the engine match the published bytes.

Provenance and license

Built by the GLM-5.3-Flash single-Spark release program, 2026-09-16/18 UTC, from turboderp/GLM-5.3-Flash-exl3 @ 51058cd551c7e570d87bd32a4adee720edce2349. License MIT, © 2026 Z.AI Co., Ltd, carried from the base; this artifact adds no new licensing terms.