GLM-5.2-Vision-NVFP4

GLM-5.2-Vision (NVFP4)

GLM-5.2 with sight. A vision-language model that bolts the MoonViT vision encoder from Kimi-K2.6 onto GLM-5.2 through a trained PatchMerger projector.

GLM-5.2 is a strong open reasoning model with no vision input. This checkpoint adds it, without touching a single GLM weight: the text backbone and the vision tower are both frozen and byte-identical to their upstream releases. The only newly-trained parameters are the 49.5M-parameter projector that maps MoonViT's 1152-dim patch embeddings into GLM's 6144-dim token space.

Quickstart

SGLang needs a small out-of-tree plugin because Glm5vForConditionalGeneration is not yet an upstream architecture. It ships inside this repo, so there is nothing else to clone:

uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
  --include 'plugins/*' --local-dir ./glm5v
uv pip install ./glm5v/plugins

SGLang

export SGLANG_EXTERNAL_MODEL_PACKAGE=sglang_glm5v
export SGLANG_EXTERNAL_MM_PROCESSOR_PACKAGE=sglang_glm5v
export SGLANG_EXTERNAL_MM_MODEL_ARCH=Glm5vForConditionalGeneration
python -m sglang_glm5v.patch

8×B200 — full 1M context

python -m sglang.launch_server \
  --model-path baseten/GLM-5.2-Vision-NVFP4 --trust-remote-code \
  --tp-size 8 \
  --quantization modelopt_fp4 \
  --disable-shared-experts-fusion --disable-flashinfer-autotune \
  --attention-backend dsa --mm-attention-backend sdpa \
  --kv-cache-dtype fp8_e4m3 --page-size 64 \
  --mem-fraction-static 0.85 \
  --context-length 1048576 \
  --reasoning-parser glm45 --tool-call-parser glm47 \
  --served-model-name glm-5.2-vision \
  --port 30000

4×B200 — 256k context

Same command with --tp-size 4, a higher memory fraction, and a smaller context:

python -m sglang.launch_server \
  --model-path baseten/GLM-5.2-Vision-NVFP4 --trust-remote-code \
  --tp-size 4 \
  --quantization modelopt_fp4 \
  --disable-shared-experts-fusion --disable-flashinfer-autotune \
  --attention-backend dsa --mm-attention-backend sdpa \
  --kv-cache-dtype fp8_e4m3 --page-size 64 \
  --mem-fraction-static 0.90 \
  --context-length 262144 \
  --reasoning-parser glm45 --tool-call-parser glm47 \
  --served-model-name glm-5.2-vision \
  --port 30000

Query it

Standard OpenAI multimodal messages deliver the image as image_url:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="none")
r = client.chat.completions.create(
    model="glm-5.2-vision",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://ultralytics.com/images/bus.jpg"}},
        {"type": "text", "text": "Describe this image in detail."},
    ]}],
    temperature=1.0, top_p=0.95, max_tokens=512,
)
print(r.choices[0].message.content)

GLM-5.2 is a reasoning model: with --reasoning-parser glm45, the chain of thought arrives in message.reasoning_content and the answer in message.content.

Deploy on Baseten

The repository includes ready-to-push Truss configs. The only credential you need is an API key for your own Baseten account; no Hugging Face token or pre-created Baseten secret is required.

1. Install uv and create a Baseten API key.
2. Export the key, download the small Truss directory, and deploy one of the two configurations:

export BASETEN_API_KEY="your-baseten-api-key"
uvx truss login --api-key "$BASETEN_API_KEY" --remote baseten --non-interactive

uvx --from huggingface-hub hf download baseten/GLM-5.2-Vision-NVFP4 \
  --include 'truss/*' --local-dir ./glm5v
cd glm5v/truss

# Recommended starting point: 4×B200 and 256k context.
uvx truss push --remote baseten --config config_nvfp4_4gpu.yaml --wait --output json

# Or use 8×B200 for the full 1M-token context.
# uvx truss push --remote baseten --config config_nvfp4.yaml --wait --output json

The command creates a new model and published deployment in your Baseten account and prints JSON containing model_id, model_version_id, predict_url, and logs_url. It does not promote the deployment to production.

Set PREDICT_URL to the returned predict_url, then query the model:

export PREDICT_URL="https://model-...api.baseten.co/deployment/.../predict"

curl -fsS "$PREDICT_URL" \
  -H "Authorization: Api-Key $BASETEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2-vision",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://ultralytics.com/images/bus.jpg"}},
        {"type": "text", "text": "Describe this image in detail."}
      ]
    }],
    "max_tokens": 512,
    "temperature": 1.0,
    "top_p": 0.95
  }'

The first deployment downloads about 466 GB of weights and initializes SGLang, so startup can take several minutes.

License

MIT, following both parents: GLM-5.2 (MIT) and Kimi-K2.6 (Modified MIT). The projector weights are released under MIT. Redistributed upstream weights remain under their original terms.

Acknowledgements

Built on Z.ai's GLM-5.2 and Moonshot AI's Kimi-K2.6. Neither team was involved in this work; please do not direct issues with this checkpoint to them.