GLM-4.7-Flash-MTP-bf16

GLM-4.7-Flash-MTP-bf16

The trained multi-token-prediction (MTP / nextn) layer of zai-org/GLM-4.7-Flash, split into a standalone MLX drafter checkpoint. This is not a standalone language model — it is a single-layer draft head that predicts one token ahead from a target model's hidden states, for speculative decoding against GLM-4.7-Flash.

GLM-4.7-Flash ships this layer inside the full checkpoint at model.layers.47.*; MLX conversions of the base model strip it (sanitize drops layers past num_hidden_layers), so quantized community conversions do not carry it. This repo preserves it in bf16, revision-pinned.

Provenance

• Source: zai-org/GLM-4.7-Flash, revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 (MIT). All weights are Z.ai's trained parameters, unmodified except for the layout transforms below.

• Tool: the glm4_moe_lite_mtp drafter split from mlx-vlm's speculative/drafters convention (Blaizzy/mlx-vlm#1570):

  python -m mlx_vlm.speculative.drafters.glm4_moe_lite_mtp.split \
    --model zai-org/GLM-4.7-Flash \
    --revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 \
    --output GLM-4.7-Flash-MTP-bf16
  

  Only the 3 (of 48) source shards holding the nextn tensors are read.

• Checksum:

  file	sha256
  model.safetensors	bdb948cbec1810d89910cc99dfbc85864acde45241337702b555e2a721254e7b

Format

model_type: glm4_moe_lite_mtp, block_size: 2 (num_nextn_predict_layers + 1), untied embeddings; the source text config is nested under text_config. 24 tensors, flat post-sanitize layout:

• dedicated nextn embed_tokens and untied lm_head (GLM's nextn head is not tied to the target, unlike DeepSeek/Qwen MTP)
• enorm / hnorm / eh_proj projections
• one MLA attention block in absorbed form (kv_b_proj split into embed_q / unembed_out)
• 64-expert MoE stacked into switch_mlp + shared expert; the noaux_tc router correction bias is kept fp32 and the router gate full precision (casting them breaks routing)

Consumers

• vllm-project/vllm-metal#484 / #485 — native MTP speculative decoding for GLM-4.7-Flash on Apple Silicon (reference integration this format feeds; design thread in #482)
• mlx-vlm's speculative-decoding runtime, once a glm4_moe_lite backbone lands there

Measured offline acceptance of this head replaying real target hidden states: ~0.806 mean over chat prompts (methodology and end-to-end results in the vllm-metal links above).

A 4-bit variant is at samithaj/GLM-4.7-Flash-MTP-4bit.