GLM-5.3-Flash-EXL3-2.05-One-Spark

GLM-5.3-Flash EXL3 2.05 — One DGX Spark

A reproducible, production-capable deployment of GLM-5.3-Flash on one NVIDIA DGX Spark, using:

• Turboderp's 2.05-bpw EXL3 checkpoint
• Inco AI's DFlash2 drafter (K5 default)
• A TP1 ARM64/SM121 vLLM runtime derived from MiaAI's two-Spark recipe

64.1 tok/s structured C1 (K7) · 29.9 tok/s prose / 40.1 tok/s code (K5 default) · 181.9 tok/s C4 active-stream aggregate · 262K context

This repository does not contain model weights

This is a deployment/runtime discovery page, not a new model or quant. The software, Docker recipe, benchmark harnesses, and raw evidence live in the linked GitHub repository. Target and draft weights download directly from their original publishers.

Measured results

One DGX Spark, TP1, EXL3 2.05 bpw, DFlash2 K7 (default at collection time; shipped default is now K5), FP8 KV, thinking disabled:

C4 disclosure: 181.944 tok/s is the sum of each stream's active decode rate, matching MiaAI's reporting convention. Because the current scheduler stages admission, strict full-batch wall throughput is 91.475 tok/s. Both numbers are published intentionally.

Speculative depth (K) sweep

Default num_speculative_tokens moved from 7 to 5 on 2026-09-03. Spec decode is lossless; K only changes speed. Temperature 0, thinking off, 400 tokens, C1:

Use K=8 for list/JSON-heavy workloads (ONE_SPARK_K=8). Raw data in the GitHub repo under benchmarks/raw/k-sweep-20260903.

Long-context results

Every cold request had zero prefix-cache hits; every warm checksum response was correct.

Quantization Analysis

This deployment uses Turboderp's GLM-5.3-Flash EXL3 2.05-bpw quant, revision 51058cd551c7e570d87bd32a4adee720edce2349. The exact checkpoint is 85.23 GB (79.38 GiB).

An independent full-vocabulary measurement of this exact revision against BF16 teacher logits reported:

The measurement used 25 windows, the full 154,880-token vocabulary, teacher forcing, FP64 accumulation, and two cold runs with identical results. The checkpoint was quantized from the official FP8 release; the measurement reference is BF16. Machine-readable summary: quantization-analysis.json.

Links

• GitHub source and instructions: https://github.com/gitcommit90/glm-5.3-one-spark
• Hugging Face Collection: https://huggingface.co/collections/gitcommit90/glm-53-one-spark-6a98b70df9981ae425acbc05
• Prebuilt ARM64 runtime image: ghcr.io/gitcommit90/glm-5.3-one-spark:general23
• Target checkpoint: https://huggingface.co/turboderp/GLM-5.3-Flash-exl3/tree/2.05bpw
• DFlash2 checkpoint: https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2
• Z.ai base model: https://huggingface.co/zai-org/GLM-5.3-Flash

Credits and licensing

The runtime is derived from Mia's AI Lab's MIT-licensed two-Spark recipe and substantially adapted for TP1, full-model 2.05-bpw mul1 EXL3, ARM64/SM121, and one-Spark memory limits.

Credits: Z.ai / GLM-5 Team, Turboderp and ExLlamaV3, Inco AI, Mia's AI Lab, and the vLLM contributors.

DFlash2 is CC BY-NC-ND 4.0 for research/evaluation. It is not bundled here. Commercial users must obtain appropriate licensing from Inco AI.