GLM-5.3-DSpark

GLM-5.3-DSpark

This repository releases a DSpark draft model for GLM-5.3. It is a speculator, not a standalone LLM: pair it with a GLM-5.3 target (BF16 or FP8) in SGLang or vLLM.

• Draft model: AlayaNeW/GLM-5.3-DSpark
• Target model: zai-org/GLM-5.3 / zai-org/GLM-5.3-FP8
• Algorithm: DSpark (parallel block draft + Markov head + confidence head)

Architecture

Quick Start

Requires a recent SGLang or vLLM build with native DSpark support. This draft is for GLM-5.3 only.

Serve with SGLang

export SGLANG_ENABLE_SPEC_V2=1

sglang serve \
  --model-path zai-org/GLM-5.3-FP8 \
  --trust-remote-code \
  --tp-size 8 \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --context-length 202752 \
  --mem-fraction-static 0.83 \
  --cuda-graph-max-bs 256 \
  --max-running-requests 128 \
  --disable-shared-experts-fusion \
  --host 0.0.0.0 \
  --port 8100 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path AlayaNeW/GLM-5.3-DSpark \
  --speculative-num-draft-tokens 9 \
  --speculative-dspark-block-size 8

--speculative-num-draft-tokens is block size + 1.

Serve with vLLM

export VLLM_USE_V2_MODEL_RUNNER=1

vllm serve zai-org/GLM-5.3-FP8 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm45 \
  --enable-prefix-caching \
  --max-model-len 202752 \
  --max-num-seqs 256 \
  --gpu-memory-utilization 0.90 \
  --max-cudagraph-capture-size 256 \
  --host 0.0.0.0 \
  --port 8100 \
  --served-model-name GLM-5.3 \
  --speculative-config '{
    "method": "dspark",
    "model": "AlayaNeW/GLM-5.3-DSpark",
    "num_speculative_tokens": 8,
    "draft_sample_method": "probabilistic",
    "kv_cache_dtype": "auto"
  }'

The draft is bf16 dense. Keep the draft KV cache dtype on auto so it does not inherit the target's FP8 MLA KV cache.

Call the server

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8100/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="GLM-5.3",
    messages=[{"role": "user", "content": "Write a Python function that checks if a number is prime."}],
    max_tokens=1024,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(resp.choices[0].message.content)

Evaluation

• Runtime: SGLang on 8× NVIDIA H200 (TP8)
• Target: zai-org/GLM-5.3-FP8
• Speculation block size: 8 (--speculative-dspark-block-size 8, --speculative-num-draft-tokens 9)
• Sampling: temperature 0.7, concurrency 10, max generation 4096
• Context-length numbers are the mean of 3 runs (50 prompts, SPEED-Bench low-entropy throughput)
• Qualitative numbers are one run on SPEED-Bench categories (80 prompts each)

Acceptance length is the per-request mean of completion tokens divided by verification steps. Higher is better. Per-position rates are percent.

Speed-Bench: Low Entropy

Speed-Bench: Qualitative

Citation

@misc{glm53dspark2026,
  title={GLM-5.3-DSpark},
  year={2026},
  howpublished={\url{https://huggingface.co/AlayaNeW/GLM-5.3-DSpark}}
}