Downloads · 30 days
57
31% of all-time downloads
GLM-5.3-NVFP4
underlabs/GLM-5.3-NVFP4
GLM-5.3-NVFP4 is a machine learning model from underlabs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
NVFP4 quantization of GLM-5.3 (Z.ai), a 745B-parameter MoE model, with the MTP draft layer also quantized to a ModelOpt NVFP4 weight-only representation for MTP speculative decoding.
All-time downloads
182
Public
Parameters
386B
457 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors451 GB · 100%
Parameter types
How the weights are stored.
U8367B · 95%
Base models
Model card
From the Hugging Face model README
GLM-5.3-NVFP4
NVFP4 quantization of GLM-5.3 (Z.ai), a 745B-parameter MoE model, with the MTP draft layer also quantized to a ModelOpt NVFP4 weight-only representation for MTP speculative decoding.
Model
| Property | Value |
|---|---|
| Base model | zai-org/GLM-5.3 |
| Immediate source checkpoint | Inferact/GLM-5.3-NVFP4 |
| Architecture | GlmMoeDsaForCausalLM (MoE) |
| Target model quantization | ModelOpt NVFP4 W4A4, inherited unchanged from Inferact |
| MTP quantization | ModelOpt NVFP4 W4A4 |
| MTP layers | 1 (num_nextn_predict_layers=1) |
| Native context length | 1,048,576 tokens |
Checkpoint differences
Only the MTP draft layer differs from the upstream Inferact checkpoint. The target-model shards were not re-quantized or otherwise modified.
| Component | Inferact checkpoint | This checkpoint |
|---|---|---|
| Target model weights | ModelOpt NVFP4 | ModelOpt NVFP4, unchanged |
| Target model execution format | W4A4 | W4A4 |
| MTP weights | BF16 | NVFP4 weight-only |
| MTP execution format | BF16 | W4A4 |
| MTP activation quantization | None | Dynamic (per-token) |
| Number of MTP layers | 1 | 1 |
| MTP storage on disk | 19.9 GB across 5 shards | 6.0 GB across 5 shards |
| MTP speculative decoding | Supported | Measured with 5 draft tokens |
MTP speculative decoding works with num_speculative_tokens=5. Measured
with vLLM's spec-decode counters on a 4×H200 (TP4) deployment:
| Scenario | Tokens per decode step | Draft acceptance | Acceptance by position (1→5) |
|---|---|---|---|
| Code, c=1, temp 0 | 3.76 | ~55% | 57 / 17 / 8 / 7 / 6 % |
Usage
Example vLLM command:
vllm serve underlabs/GLM-5.3-NVFP4 \
--tensor-parallel-size 4 \
--trust-remote-code \
--enable-expert-parallel \
--kv-cache-dtype fp8_ds_mla \
--speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
--reasoning-parser glm45 \
--tool-call-parser glm47 \
--enable-auto-tool-choice
Credits
- Z.ai — base model, weights, and license
- Inferact — NVFP4 quantization of the target model:
Inferact/GLM-5.3-NVFP4 - underlabs — weight-only quantization of the MTP layer and checkpoint assembly
License
This checkpoint is distributed under the same license as the base model. See LICENSE for the complete terms, including the Model-as-a-Service provisions.
- Type
- glm_moe_dsa
- License
- other
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026