GLM-4.7-Flash-Coder-GGUF

GLM-4.7-Flash-Coder GGUF

Community GGUF quantizations of whitecircle/GLM-4.7-Flash-Coder.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;"> ☕ If this GGUF made your day easier, a coffee would make mine.<br> <a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br> I build and test these releases myself. Your coffee helps keep me going.<br> Thank you for supporting this work. </div>

About GLM-4.7-Flash-Coder

GLM-4.7-Flash-Coder is an agentic coding fine-tune of zai-org/GLM-4.7-Flash. The upstream model card describes a 30B Mixture-of-Experts model trained on software-engineering trajectories, with tool use, reasoning, and concurrent tool-call behavior. The upstream training configuration uses a 75k-token sequence length; its model configuration declares a 202,752-token maximum position limit. This release does not claim long-context behavior beyond the runtime validation documented below.

The model uses the GLM-4.7 Flash chat template and XML-style tool calls. See the official upstream model card for the original model, training, and evaluation details.

This repository contains quantizations only. No training or fine-tuning was performed here.

Fidelity measurements

These are next-token fidelity measurements against the locked BF16 GGUF, not a task benchmark. Every published quantization was evaluated on the separate wiki.valid.raw hold-out from WikiText-2, using 8 sequential chunks at context 2048 with llama-perplexity from llama.cpp commit 4a3635c32fc9f044c2bde9ebeabf50c7e1ec5991. The BF16 reference used partial GPU 0 offload (-ngl 10); quantized files used GPU 0 with -ngl 20 for the larger files and -ngl 99 for the smaller files. Values are aggregate means across the 8 evaluated chunks. The calibration text used for the imatrix was kept separate from this hold-out.

Lower KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate behavior closer to BF16. The two bold rows are measured memory/fidelity candidates based on file size and hold-out fidelity; no precise VRAM claim is made here. Quality can vary by workload, prompt, and runtime settings.

See the compact quality summary for machine-readable values and the reproducibility manifest for corpus hashes and the exact evaluation profile.

Quick start

The first validated artifact is GLM-4.7-Flash-Coder-Q8_0.gguf; run it with llama.cpp:

./llama-cli \
  -m GLM-4.7-Flash-Coder-Q8_0.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

The included chat_template.jinja should be used for chat and tool-call serialization. Tool calling and reasoning claims are limited to the validation profiles recorded in the reproducibility manifest.

Reproducibility and validation

Artifacts are being released incrementally. Each quantization is generated directly from the locked BF16 GGUF source, validated with the local llama.cpp runtime, checksummed, and uploaded in a separate Hub commit. The baseline ladder is:

Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M, Q2_K, IQ2_XS, IQ1_M, Q1_0.

Additional direct-from-BF16 profile variants released here are Q5_K_S, Q4_K_S, IQ4_XS, Q3_K_L, Q3_K_S, and IQ3_M.

The reproducibility manifest records the upstream revision, source hashes, converter/runtime revision, calibration input, quantization commands, validation profile, and publication status. The model-specific calibration text and importance matrix are included under reproducibility/. The public SHA256SUMS.txt records checksums for published files. Raw conversion, calibration, quantization, smoke-test, and benchmark logs remain local under reports/.

Q8_0, Q6_K, Q5_K_M, and Q4_K_M have passed CPU and GPU 0 load/generate smoke tests with the included chat template. Q5_K_S passed CPU smoke and a GPU 0 partial-offload smoke at -ngl 20 with context 2048; a full-offload attempt exceeded the available GPU 0 memory and is not claimed as a full-GPU profile. Q4_K_S, IQ4_XS, Q3_K_L, Q3_K_S, and IQ3_M passed CPU and full GPU 0 smoke at -ngl 99 with context 2048. Q3_K_M, Q2_K, IQ2_XS, and IQ1_M have passed CPU smoke testing and are published; GPU 0 smoke was not run for these lower formats. Q1_0 was attempted but is omitted because this runtime has no fallback for a non-block-aligned tensor; no invalid artifact is published.

The fidelity table above compares every published quantization with the same BF16 reference. Raw conversion, fidelity, quantization, smoke-test, and benchmark logs remain local under reports/; the public package contains only the compact quality summary and reproducibility metadata needed to reproduce the release.

License and attribution

The upstream model card declares the MIT license; see LICENSE and the upstream model card.

These are community GGUF quantizations, not an official whitecircle/GLM-4.7-Flash-Coder release or endorsement.