GLM-4.7-Flash-MTP-4bit

GLM-4.7-Flash-MTP-4bit

The trained multi-token-prediction (MTP / nextn) layer of zai-org/GLM-4.7-Flash, split into a standalone MLX drafter checkpoint and quantized to 4-bit. This is not a standalone language model — it is a single-layer draft head that predicts one token ahead from a target model's hidden states, for speculative decoding against GLM-4.7-Flash (pairs with mlx-community/GLM-4.7-Flash-4bit).

GLM-4.7-Flash ships this layer inside the full checkpoint at model.layers.47.*; MLX conversions of the base model strip it (sanitize drops layers past num_hidden_layers), so quantized community conversions do not carry it. This repo preserves it, revision-pinned.

Provenance

• Source: zai-org/GLM-4.7-Flash, revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 (MIT). All weights are Z.ai's trained parameters, unmodified except quantization and the layout transforms below.

• Tool: the glm4_moe_lite_mtp drafter split from mlx-vlm's speculative/drafters convention (Blaizzy/mlx-vlm#1570):

  python -m mlx_vlm.speculative.drafters.glm4_moe_lite_mtp.split \
    --model zai-org/GLM-4.7-Flash \
    --revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67 \
    --output GLM-4.7-Flash-MTP-4bit \
    --q-bits 4 --q-group-size 64
  

  Only the 3 (of 48) source shards holding the nextn tensors are read.

• Checksum:

  file	sha256
  model.safetensors	cc2a9750a6328a68b1758502a47f5d286cbc96c26859210fe7c751bbe6d328ba

Format

model_type: glm4_moe_lite_mtp, block_size: 2 (num_nextn_predict_layers + 1), untied embeddings, affine quantization (bits: 4, group_size: 64); the source text config is nested under text_config. 54 tensors, flat post-sanitize layout:

• dedicated nextn embed_tokens and untied lm_head (GLM's nextn head is not tied to the target, unlike DeepSeek/Qwen MTP)
• enorm / hnorm / eh_proj projections
• one MLA attention block in absorbed form (kv_b_proj split into embed_q / unembed_out)
• 64-expert MoE stacked into switch_mlp + shared expert
• not quantized: norms, the router gate weight, and the noaux_tc router correction bias (kept fp32) — casting or quantizing them breaks routing

Consumers

• vllm-project/vllm-metal#484 / #485 — native MTP speculative decoding for GLM-4.7-Flash on Apple Silicon (reference integration this format feeds; design thread in #482)
• mlx-vlm's speculative-decoding runtime, once a glm4_moe_lite backbone lands there

Measured offline acceptance of this head replaying real target hidden states: ~0.806 mean over chat prompts; a 4-bit head forward measured ~6.7% of a target decode step on M3 (methodology and end-to-end results in the vllm-metal links above).

An unquantized variant is at samithaj/GLM-4.7-Flash-MTP-bf16.