GLM-5.2-speedy-colibri-nvfp4

GLM-5.2 · SpeedyColibri NVFP4 container

A pre-converted colibrì container for the GLM-5.2 744B MoE, ready to serve on a single DGX Spark (GB10) with SpeedyColibri. Downloading this repo lets you skip the ~25-minute local conversion — the engine detects it as a container and serves it directly.

What this is

• GLM-5.2 (glm_moe_dsa, 744B, 256 experts × top-8, 78 layers) in colibrì's own streaming-MoE container format.
• Routed experts: NVFP4 — 4-bit block-scaled (e2m1 + per-16 ue4m3 block scale + per-tensor global), ~0.56 B/weight, ~403 GB. Streamed from disk on demand.
• Resident weights (attention / dense / shared / embeddings): int8 (~19 GB, kept in RAM).
• Includes the MTP speculative-decoding head (has_mtp=true) — opt-in via DRAFT=n.

⚠️ Not a stock checkpoint

This is not loadable by transformers, vLLM, or llama.cpp. It is the colibrì engine's own on-disk container (out-*.safetensors + mtp-*.safetensors with .qs/.g scale sidecars). Use it only with SpeedyColibri / colibrì.

Usage

# One command — downloads this container and serves an OpenAI-compatible API on :8080
docker/run-dgx.sh --model Kanposer/GLM-5.2-speedy-colibri-nvfp4 serve 8080

Or from a clone of github.com/GriffinPilz/SpeedyColibri, by registry name — serve.sh downloads the container when it is missing, and fetch.sh is that download on its own:

scripts/fetch.sh glm-5.2            # 403 GB — budget the disk
scripts/serve.sh glm-5.2 8080

After download it loads the ~19 GB resident weights in ~18s and is ready; the NVFP4 experts stream from disk per token. See the SpeedyColibri README for tuning (COLI_RAM_GB, DRAFT, COLI_CTX, …) and the measured performance record.

Measured on one DGX Spark (GB10, 121.7 GiB), 2026-08-09

Single sequence, greedy, 512-token prompt. Median of repeated runs on one build, gated on token identity — a "faster" number that changed the output fails loudly instead of being reported as a win.

The 24-token decode window is mostly warm-up at ~1 tok/s, so it understates this model: run out to 100 tokens and steady state measures ~1.23–1.33 tok/s. The window is kept at 24 for comparability with every previously recorded figure.

prefill is a draw from a range on this box, not a fixed figure. The same binary and prompt read 62.0, 65.1, 66.9, 72.4 and 80.1 s across two days — 8.3 down to 6.4 tok/s — with no code change. A reboot resets it to the fast end and it degrades within ~20 minutes of repeated large-model streaming; GLM moves 403 GB through 121 GB of RAM, so page-cache and fragmentation state dominates. The figure above is one draw near the middle of that range. Treat roughly +-15% as the honest width, and do not compare it against a figure measured in a different session.

decode is the noisiest column here. The eight reps behind that median spanned 0.98-1.07 tok/s forward-only in one session — a 9% spread with no code change. Treat a single suite as one draw, and do not read a difference against an earlier revision of this card as a speedup.

Method, caveats and the full fleet table: SpeedyColibri README.

Provenance & license

MIT. This is a derivative of zai-org/GLM-5.2 (MIT), quantized to NVFP4 by NVIDIA as nvidia/GLM-5.2-NVFP4 (MIT), then requantized/repacked into the colibrì container format by SpeedyColibri (no additional loss — the modelopt NVFP4 experts are carried through directly). All credit for the model weights is theirs; this repo only changes the on-disk layout for streaming inference. Redistributed under MIT with attribution.