GLM-5.3-speculator.dspark

RedhatAI/GLM-5.3-speculator.dspark

Model Overview

• Model Architecture: DSparkDraftModel
  • Target Model Architecture: GlmMoeDsaForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Speculative Decoding Algorithm: DSpark
  • Draft Model: 3-layer Qwen3-style backbone
  • Maximum Draft Length: 8 tokens
  • Auxiliary Target Layers: 2, 20, 39, 58, and 75
  • Markov Head: Vanilla, rank 256
  • Confidence Head: Enabled with Markov features
• Maximum Context Length: 1,048,576 tokens
• Release Date: 2026-09-16
• Version: 1.0
• Model Developers: Red Hat AI

This is a DSpark speculator model for GLM-5.3. It was trained using the Speculators library (version 0.7.0.dev108).

DSpark extends DFlash with a Markov head for modeling intra-block token dependencies and a confidence head for predicting per-position acceptance. The 3-layer draft model consumes auxiliary hidden states from layers 2, 20, 39, 58, and 75 of the target model and proposes up to 8 tokens per decoding step.

The model supports GLM-5.3's native context length of 1,048,576 tokens. In the 524K–1M-token MRCR evaluation bucket, it achieved a 41.16% token acceptance rate and an average accepted length of 4.293 tokens.

Training Details

This model was trained using the Speculators library. It was warm-started from a GLM-5.3 D-Spark checkpoint and trained for one complete epoch on 1,811,739 prepared 8,192-token sequences. Hidden-state extraction ran through vLLM while four FSDP ranks trained the draft model.

<details> <summary>Commands</summary>

Prepare data

python scripts/prepare_data.py \
  --model zai-org/GLM-5.3 \
  --data ./data.jsonl \
  --output ./runs/glm53_dspark/data \
  --seq-length 8192

Launch hidden-state extraction

python scripts/launch_vllm.py zai-org/GLM-5.3 \
  --target-layer-ids 2 20 39 58 75 \
  -- --tensor-parallel-size 4 --max-model-len 8193 --port 8000

Launch training

torchrun --standalone --nproc-per-node 4 scripts/train.py \
  --verifier-name-or-path zai-org/GLM-5.3 \
  --from-pretrained <GLM-5.3-DSPARK-WARMSTART> \
  --data-path ./runs/glm53_dspark/data \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./runs/glm53_dspark/checkpoints \
  --epochs 1 --lr 1e-4 --total-seq-len 8192 \
  --speculator-type dspark --max-anchors 1024 \
  --target-layer-ids 2 20 39 58 75 \
  --loss-fn '{"ce":0.1,"tv":0.9}' \
  --confidence-head-alpha 1.0 --fsdp-shard \
  --on-missing generate --on-generate delete --checkpoint-freq 0.01

The released checkpoint completed 88,226 optimizer steps.

</details>

Model Specifications

Deployment

vllm serve zai-org/GLM-5.3 \
  --tensor-parallel-size 8 \
  --speculative-config '{"model":"RedHatAI/GLM-5.3-speculator.dspark","num_speculative_tokens":8,"method":"dspark"}'

For further optimization, consider swapping the GLM-5.3 checkpoint for its NVFP4 quantized counterpart, RedHatAI/GLM-5.3-NVFP4:

vllm serve RedHatAI/GLM-5.3-NVFP4 \
  --tensor-parallel-size 4 \
  --speculative-config '{"model":"RedHatAI/GLM-5.3-speculator.dspark","num_speculative_tokens":8,"method":"dspark"}'

Acceptance Rates

Per-position token acceptance rates across datasets:

Long-context acceptance

Acceptance on MRCR 2-needle prompts by input context length.

Performance Eval Against MTP

In the graph, interactivity is the median per-request output-token rate, aggregate throughput is total completed output tokens divided by measurement duration, and each label is completed requests divided by measurement duration.