GLM-5.3-Flash-GGUF

GLM-5.3-Flash-GGUF

Community llama.cpp GGUFs of Z.ai's GLM-5.3-Flash (Glm5NextForConditionalGeneration, 320B-A18B, native multimodal).

This pack is the first working GGUF conversion of GLM-5.3-Flash. The convert registers GGUF arch glm5next and maps the 5.3 hybrid end-to-end: KDA (gated delta-net), NoPE sparse MLA + lightning indexer + IndexPool, mHC, 288-expert MoE + shared expert, MTP / NextN, and the vision tower as mmproj. It is a 5.3 convert, not a GLM-5.2 GlmMoeDsa class rename (that path only covers the 11 MLA blocks and drops the 34 KDA layers).

llama-quantize on that same glm5next build wrote the K-quants below (Q2_K 2.91 BPW, Q4_K_M 4.81 BPW). Run these GGUFs on vcruz305/llama.cpp main / glm5next-mtp (ea37b8b): Unsloth glm5next graph plus NextN graph_mtp, with the trunk emitting t_h_nextn so --spec-type draft-mtp can accept. ggml-org drafts #27752, #27754, #27773 load the weights; MTP decode is on this fork.

Sibling NVFP4 pack: vcruz305/GLM-5.3-Flash-NVFP4.

Convert method

HF class Glm5NextForConditionalGeneration / glm5_next with nested text_config and prefix model.language_model.*.

Patched llama.cpp convert (not upstream main):

1. New arch enum GLM5_NEXT / glm5next in gguf-py + C++ llama-arch.
2. Glm5NextModel in conversion/glm.py, registered on Glm5NextForConditionalGeneration.
3. Tensor map reuses 5.2 MoE / MLA / MTP names where they match, and adds 5.3-only tensors: KDA (ssm_conv1d_{q,k,v}, ssm_f_a/b, ssm_g_a/b, A_log/ssm_a, dt), IndexPool (indexer_compressor_ape, indexer_compressor_gate), mHC (hc_attn_*, hc_ffn_*).
4. Hybrid head_count_kv per layer (0 on KDA, 1 on sparse MLA). qk_rope_head_dim=0 / NoPE on the MLA blocks.
5. MTP lives at model.layers.{n_layer} (eh_proj, enorm, hnorm, shared_head). Convert keeps it; F16 block_count=46 includes NextN.
6. Vision is a second GGUF via --mmproj (v.blk.* 24 blocks, patch embed, merger) — same recipe as other llama.cpp VLMs.

Toy fixture used first: inference-optimization/GLM-5.3-Flash-0.1B-A0.1B (same class, 5 layers / 8 experts / 2 vision blocks). Full convert is the 320B-A18B BF16, 1412 text tensors.

Files

Hub lists a file only after its upload commit.

F16 source (local, unsplit): 597.587 GiB / 16.00 BPW / 1412 tensors. Split with llama-gguf-split --split-max-size 400G because a single 641.65 GB file is over the Hub per-file limit.

llama-quantize on this build (d7a2074): 68 of 1412 tensors fall back (KDA ssm_f_b / ssm_g_b ncols 128, not divisible by 256). Experts stay on the requested K type. Output tensor is q6_K on the K mixes.

More K rungs (Q3_K_*, Q5_K_*, Q6_K) follow the same F16.

Quick start

Q2_K MTP on CUDA. Pin the fork to ea37b8b (main / glm5next-mtp). Proof tool is llama-speculative-simple -lv 4. Look for n_accept / accept in the log.

hf download vcruz305/GLM-5.3-Flash-GGUF GLM-5.3-Flash-Q2_K.gguf --local-dir GLM-5.3-Flash-GGUF

git clone https://github.com/vcruz305/llama.cpp.git
cd llama.cpp
git checkout ea37b8bb053025f1f1e86e9e014b6cde8a98caa1

cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
# DGX Spark / GB10:
# cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121 -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-speculative-simple -j

./build/bin/llama-speculative-simple \
  -m ../GLM-5.3-Flash-GGUF/GLM-5.3-Flash-Q2_K.gguf \
  -p "The capital of France is" \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -fit on -c 2048 -fitc 2048 -n 64 \
  --temp 0 --top-k 1 --seed 42 --perf \
  -lv 4

Measured on one NVIDIA GB10 Spark, Q2_K, greedy, -c 2048, 1281-token prompt: accept 68.182% (n_drafted=66, n_accept=45), encode 279.4 tok/s, decode 27.17 tok/s.

Q4_K_M + mmproj:

hf download vcruz305/GLM-5.3-Flash-GGUF GLM-5.3-Flash-Q4_K_M.gguf --local-dir GLM-5.3-Flash-GGUF
hf download vcruz305/GLM-5.3-Flash-GGUF mmproj-GLM-5.3-Flash-F16.gguf --local-dir GLM-5.3-Flash-GGUF

That tree is Unsloth glm5next/upstream plus NextN graph_mtp (dense NoPE MLA on blk.45, plain KV cache for il >= n_layer). GGUFs keep the MTP tensors (nextn_predict_layers=1).

Quantize was llama-quantize on the glm5next convert build.

Official Z.ai introduction, serve links, footnotes, and citation are below, unchanged. Those benchmark figures are Z.ai's on the official release, not scores from these GGUFs.

---

Original model card (Z.ai GLM-5.3-Flash)

GLM-5.3-Flash

<div align="center"> <img src=https://raw.githubusercontent.com/zai-org/GLM-5/refs/heads/main/resources/logo.svg width="15%"/> </div> <p align="center"> 👋 Join our <a href="https://raw.githubusercontent.com/zai-org/GLM-5/refs/heads/main/resources/wechat.png" target="_blank">WeChat</a> or <a href="https://discord.gg/QR7SARHRxK" target="_blank">Discord</a> community. <br> 📖 Check out the GLM-5.3-Flash <a href="https://z.ai/blog/glm-5.3-flash" target="_blank">blog</a> and GLM-5 <a href="https://arxiv.org/abs/2602.15763" target="_blank">Technical report</a>. <br> 📍 Use GLM-5.3-Flash API services on <a href="https://docs.z.ai/guides/llm/glm-5.3-flash">Z.ai API Platform. </a> </p>

Introduction

We introduce GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series. With 320B total parameters and just 18B active parameters, it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.

GLM-5.3-Flash starts from a newly trained base model, with its architecture and training recipe redesigned around capability and efficiency. For the first time in the GLM series, we introduce a hybrid architecture combining sparse and linear attention, sharply reducing long-context serving costs while preserving precise long-context capabilities. The model also adopts Manifold-Constrained Hyper-Connections (mHC) to further improve scaling efficiency. Together with our latest 30T-token multimodal pre-training corpus, these changes enable GLM-5.3-Flash to deliver more intelligence with less compute.

Serve GLM-5.3-Flash Locally

GLM-5.3-Flash supports deployment with the following frameworks. Feel free to try them out:

• SGLang — see cookbook
• vLLM — see recipes
• TokenSpeed — see here
• KTransformers — see tutorial

Footnotes

• HLE w/ tools (full set): We use sampling parameters of temperature=1.0 and top_p=0.95 for evaluation, with a maximum generation length of 163,840 tokens. The evaluation is conducted with a maximum context length of 300,000 tokens, using a context management strategy. We use GPT-5.6-luna (medium) as the judge model.
• NL2Repo: We evaluated NL2Repo with temperature=1.0, top_p=1.0, and max_new_tokens=64k under 1M context. To prevent hacking, we use rule-based and a LLM-based judgement to prevent malicious behaviors (e.g., unauthorized pip or curl operations).
• DeepSWE: We run DeepSWE using the mini-swe-agent harness with temperature=0.95, top_p=1.0, timeout=6h and 400K context.
• Terminal-Bench 2.1: We evaluate in Claude Code 2.1.207 with temperature=1.0, top_p=1, max_new_tokens=65536 with 6h timeout.
• Agent’s Last Exam:
• Toolathlon Verified: We obtain all results via the official evaluation service and report pass@1 averaged over 3 independent runs.
• AutomationBench: We evaluate on AutomationBench v1.0.6, incorporating the fix for the null-type handling issue introduced in PR #13.
• GDPval-AA v2: Models are evaluated by Artificial Analysis.
• BabyVision: We use temperature=1.0, top_p=0.95, and a maximum context length of 164K tokens. We resize the input images such that their shorter side is at least 1.5K pixels, consistent with other baselines.

Citation

If you find GLM-5.3-Flash useful in your research, please cite our technical report:

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team and : and Aohan Zeng and Xin Lv and Zhenyu Hou and Zhengxiao Du and Qinkai Zheng and Bin Chen and Da Yin and Chendi Ge and Chenghua Huang and Chengxing Xie and Chenzheng Zhu and Congfeng Yin and Cunxiang Wang and Gengzheng Pan and Hao Zeng and Haoke Zhang and Haoran Wang and Huilong Chen and Jiajie Zhang and Jian Jiao and Jiaqi Guo and Jingsen Wang and Jingzhao Du and Jinzhu Wu and Kedong Wang and Lei Li and Lin Fan and Lucen Zhong and Mingdao Liu and Mingming Zhao and Pengfan Du and Qian Dong and Rui Lu and Shuang-Li and Shulin Cao and Song Liu and Ting Jiang and Xiaodong Chen and Xiaohan Zhang and Xuancheng Huang and Xuezhen Dong and Yabo Xu and Yao Wei and Yifan An and Yilin Niu and Yitong Zhu and Yuanhao Wen and Yukuo Cen and Yushi Bai and Zhongpei Qiao and Zihan Wang and Zikang Wang and Zilin Zhu and Ziqiang Liu and Zixuan Li and Bojie Wang and Bosi Wen and Can Huang and Changpeng Cai and Chao Yu and Chen Li and Chengwei Hu and Chenhui Zhang and Dan Zhang and Daoyan Lin and Dayong Yang and Di Wang and Ding Ai and Erle Zhu and Fangzhou Yi and Feiyu Chen and Guohong Wen and Hailong Sun and Haisha Zhao and Haiyi Hu and Hanchen Zhang and Hanrui Liu and Hanyu Zhang and Hao Peng and Hao Tai and Haobo Zhang and He Liu and Hongwei Wang and Hongxi Yan and Hongyu Ge and Huan Liu and Huanpeng Chu and Jia'ni Zhao and Jiachen Wang and Jiajing Zhao and Jiamin Ren and Jiapeng Wang and Jiaxin Zhang and Jiayi Gui and Jiayue Zhao and Jijie Li and Jing An and Jing Li and Jingwei Yuan and Jinhua Du and Jinxin Liu and Junkai Zhi and Junwen Duan and Kaiyue Zhou and Kangjian Wei and Ke Wang and Keyun Luo and Laiqiang Zhang and Leigang Sha and Liang Xu and Lindong Wu and Lintao Ding and Lu Chen and Minghao Li and Nianyi Lin and Pan Ta and Qiang Zou and Rongjun Song and Ruiqi Yang and Shangqing Tu and Shangtong Yang and Shaoxiang Wu and Shengyan Zhang and Shijie Li and Shuang Li and Shuyi Fan and Wei Qin and Wei Tian and Weining Zhang and Wenbo Yu and Wenjie Liang and Xiang Kuang and Xiangmeng Cheng and Xiangyang Li and Xiaoquan Yan and Xiaowei Hu and Xiaoying Ling and Xing Fan and Xingye Xia and Xinyuan Zhang and Xinze Zhang and Xirui Pan and Xu Zou and Xunkai Zhang and Yadi Liu and Yandong Wu and Yanfu Li and Yidong Wang and Yifan Zhu and Yijun Tan and Yilin Zhou and Yiming Pan and Ying Zhang and Yinpei Su and Yipeng Geng and Yong Yan and Yonglin Tan and Yuean Bi and Yuhan Shen and Yuhao Yang and Yujiang Li and Yunan Liu and Yunqing Wang and Yuntao Li and Yurong Wu and Yutao Zhang and Yuxi Duan and Yuxuan Zhang and Zezhen Liu and Zhengtao Jiang and Zhenhe Yan and Zheyu Zhang and Zhixiang Wei and Zhuo Chen and Zhuoer Feng and Zijun Yao and Ziwei Chai and Ziyuan Wang and Zuzhou Zhang and Bin Xu and Minlie Huang and Hongning Wang and Juanzi Li and Yuxiao Dong and Jie Tang},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}