GLM-5.2-SIQ-Fruit-bf16

GLM-5.2-SIQ-Fruit-bf16

The plain-BF16, CPU-runnable twin of GLM-5.2-SIQ-Fruit. It contains the same QNOISE-annealed 5.04B-parameter checkpoint—0.46B parameters active per token—but replaces the custom SIQ expert representation with ordinary BF16 tensors.

This repository loads with stock Transformers and needs no GPU. It is the reference path for model behavior and CPU integration; it does not exercise SIQ/Trellis dequantization, sparse DSA attention, low-precision KV caches, or MTP speculative decoding.

Measured CPU run

Intel Core i7-14700K, 20 Torch threads, BF16, Transformers 5.14.1; 128-token greedy decode after a 16-token warmup:

An earlier card estimated “about 10 GB RAM.” That was not an end-to-end process measurement and has been withdrawn.

Usage

import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer

model_id = "malaiwah/GLM-5.2-SIQ-Fruit-bf16"
torch.set_num_threads(20)

config = AutoConfig.from_pretrained(model_id)
assert config.rope_theta == config.rope_parameters["rope_theta"] == 500000.0

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    config=config,
    dtype=torch.bfloat16,
).eval()

inputs = tokenizer("Once upon a time", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Expected output begins with a TinyStories-style continuation about a little girl named Lily; exact wording can vary with library kernels and version.

What loads

• Exercised: dense MLA attention, GLM tokenizer, and the real 256-expert top-8 MoE weights.
• Ignored by stock Transformers: DSA indexer tensors and the MTP draft layer. They are reported as unexpected keys. This is intentional for this compatibility path; 4.57B of the 5.04B parameters load.
• Production path: use the SIQ release with a compatible b12x/SparkInfer + vLLM build to exercise Trellis experts, sparse MLA, fp8/nvfp4 KV, and MTP.

Release correction

Revision b97f91d222906142e46827a010a5caf5a9e35928 encoded the trained RoPE theta incorrectly: it retained 8,000,000 in the nested field and omitted the legacy top-level field. The 2026-08-07 correction writes 500,000 to both locations and regenerates MANIFEST.sha256. Weight shards did not change. The fail-closed assertion in the usage example protects pinned or cached stale configurations.

The export also carries the same trainer-to-serving RoPE channel permutation and MTP eh_proj input-half conversion as the SIQ artifact; those transformations preserve the trained function before stock Transformers drops the unsupported indexer/MTP modules.

Limitations

• The model is a serving proxy and CI fixture, not a general assistant.
• CPU generation uses dense attention and therefore does not validate the DSA indexer or long-context sparse-attention path.
• The measured speed and memory numbers are for the exact hardware/software setup above; they are not hardware-independent guarantees.

Reproducibility and integrity

MANIFEST.sha256 authenticates all serving files except the card and Git attributes. The exporter invocation uses FRUIT_BF16=1; source checkpoint, training recipe, gauntlet, and review evidence are documented in proxy-fruit and the base model card.

License

MIT; see LICENSE. Earlier model-card metadata incorrectly reported Apache-2.0; the packaged license file has always been MIT.