GLM-5.3-NVFP4-DFlash

GLM-5.3-NVFP4-DFlash

Paper | Github | Blog

This repository contains a DFlash draft model for modal-labs/GLM-5.3-NVFP4. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.

DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.

Quick Start

Serve with SGLang v0.5.19 or later:

python -m sglang.launch_server \
  --model-path modal-labs/GLM-5.3-NVFP4 \
  --quantization modelopt_fp4 \
  --tp-size 4 \
  --trust-remote-code \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path modal-labs/GLM-5.3-NVFP4-DFlash \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant \
  --speculative-draft-attention-backend fa4 \
  --host 0.0.0.0 \
  --port 30000

Keep the draft model unquantized. Quantizing it lowers the accept length.

License

Distributed under Z.ai's GLM-5.3 License, inherited from the target model.

Citation

If you find DFlash useful, please cite the original paper:

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}