GLM-4.7-Flash-Q8_0-GGUF

khasinski/GLM-4.7-Flash-Q8_0-GGUF

GLM-4.7-Flash-Q8_0-GGUF is a machine learning model from khasinski. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.

Q80 quantization of zai-org/GLM-4.7-Flash for use with llama.cpp.

Downloads · 30 days

6% of all-time downloads

All-time downloads

430

Public

Repo size

31.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf31.8 GB · 100%

At a glance

License: mit
Access: Public
Created: Jan 19, 2026
Updated: Jan 19, 2026
SHA: 10a1c7ff

Base models

zai-org/GLM-4.7-Flash

Model card

From the Hugging Face model README

GLM-4.7-Flash Q8_0 GGUF

Q8_0 quantization of zai-org/GLM-4.7-Flash for use with llama.cpp.

Model Details

Property	Value
Base model	zai-org/GLM-4.7-Flash
Architecture	30B-A3B MoE (DeepSeek v2)
Quantization	Q8_0
Size	~30 GB
Context length	128K tokens

Hardware Requirements

Minimum VRAM: 32 GB (single GPU)
Recommended: 56 GB (dual GPU, e.g., RTX 5090 + RTX 4090)

Usage

Basic usage with llama.cpp

llama-server -m GLM-4.7-Flash-Q8_0.gguf -ngl 99 -c 65536

Full 128K context on dual GPU

llama-server -m GLM-4.7-Flash-Q8_0.gguf \
    -ngl 99 \
    -c 131072 \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    --split-mode layer \
    --tensor-split 32,24 \
    --host 0.0.0.0 \
    --port 8080

OpenAI-compatible API

After starting the server, the API is available at:

http://localhost:8080/v1/chat/completions
http://localhost:8080/v1/completions

Quantization Notes

This model was quantized from ngxson/GLM-4.7-Flash-GGUF F16 version.

The missing deepseek2.rope.scaling.yarn_log_multiplier metadata key was added to enable quantization with llama.cpp.

Quality Comparison

Quantization	Size	Perplexity Impact
F16	56 GB	Baseline
Q8_0	30 GB	~0.1%
Q4_K_M	18 GB	~2-4%

Q8_0 provides near-lossless quality while reducing model size by ~47%.

License: mit
Created: Jan 19, 2026
Updated: Jan 19, 2026