GLM-Z1-32B-0414-int4-ov

GLM-Z1-32B-0414 — OpenVINO INT4 (OV-IR)

An OpenVINO Intermediate Representation (OV-IR) of zai-org/GLM-Z1-32B-0414, weight-compressed to INT4 (symmetric) for efficient inference on Intel Arc GPUs via the OpenVINO Model Server (OVMS) or openvino-genai.

GLM-Z1-32B-0414 is Zhipu AI's 32B reasoning model (Glm4ForCausalLM). This is a straight quantized conversion — the weights, tokenizer, and chat template are the original model's; only the numerical precision and serialization format differ.

What's in this repo

A single-graph, text-servable OV-IR:

• openvino_model.xml / openvino_model.bin — the INT4 model graph + weights (~17 GB)
• openvino_tokenizer.* / openvino_detokenizer.* — OpenVINO tokenizer IRs
• chat_template.jinja, tokenizer.json, tokenizer_config.json, config.json, generation_config.json
• graph.pbtxt, openvino_config.json — OVMS serving graph + runtime config

Conversion

optimum-cli export openvino \
  --model zai-org/GLM-Z1-32B-0414 \
  --task text-generation-with-past \
  --weight-format int4 --sym --ratio 1.0 --group-size 128

• optimum-intel 2.0.0 · openvino 2026.2.1 · nncf 3.2.0 · transformers 5.12.1
• Precision: INT4 symmetric, group size 128, ratio 1.0

Verified

Loads and generates on an Intel Arc Pro B70 (32 GB) under OVMS 2026.2. Strict json_schema constrained decoding (XGrammar) returns schema-conformant output. Coding and code-localization spot-checks pass.

Usage (OpenVINO Model Server)

Place this directory in your OVMS model repository and serve it as a text-generation LLM graph (/v3/chat/completions). The included graph.pbtxt provides the serving pipeline.

License & attribution

Inherits the MIT license of the base model, zai-org/GLM-Z1-32B-0414. Please cite the original authors (Zhipu AI / Z.ai). This repository only re-distributes a quantized OpenVINO conversion.