Downloads · 30 days
1.3K
24% of all-time downloads
groxaxo/Qwen3.5-9B-GLM5.1-Distill-v1-GGUF
Qwen3.5-9B-GLM5.1-Distill-v1-GGUF is a machine learning model from groxaxo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Qwen3.5-9B-GLM5.1-Distill-v1-GGUF is a GGUF release for llama.cpp-compatible runtimes and local inference, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible…
Downloads · 30 days
1.3K
24% of all-time downloads
All-time downloads
5.6K
Public
Repo size
84.8 GB
Likes
1
Public
Click a slice to open those files.
.gguf84.8 GB · 100%
From the Hugging Face model README
Qwen3.5-9B-GLM5.1-Distill-v1-GGUF is a GGUF release for llama.cpp-compatible runtimes and local inference, published by groxaxo.
It is intended for open-source evaluation, reproducible experimentation, and compatible local or
hosted inference workflows. The wording below is deliberately limited to what can be verified
from this repository's metadata and artifacts.
| Field | Details |
|---|---|
| Format | GGUF |
| Source / base | Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1 |
| Intended task | the task described by the included configuration and documentation |
| License | apache-2.0 |
*.gguf (13 files)Download a .gguf file that fits your available memory, then run it with a current llama.cpp
build:
llama-cli \
-m /path/to/model.gguf \
-p "Write a concise technical summary."
For vision or any-to-any models, download the matching multimodal projection file when one is provided and follow the source model's modality-specific instructions.
Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.
Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.
<!-- polished-overview:end -->All quantizations were produced using an importance matrix (imatrix) computed on 128 chunks of WikiText-2-raw-v1 for optimal quality at every bit-width.
| Variant | Size | BPW | Notes |
|---|---|---|---|
| F16 | 17.0 GB | 16.00 | Lossless half-precision |
| Q8_0 | 8.9 GB | 8.50 | Near-lossless, 8-bit round quant |
| Q6_K | 6.9 GB | 6.57 | 6-bit K-quant, excellent quality |
| Q5_K_M | 6.1 GB | 5.77 | 5-bit K-medium, recommended sweet spot |
| Q5_K_S | 5.9 GB | 5.62 | 5-bit K-small, slight size savings |
| Q4_K_M | 5.3 GB | 5.02 | 4-bit K-medium, best quality/size ratio |
| Q4_K_S | 5.0 GB | 4.77 | 4-bit K-small, good balance |
| IQ4_XS | 4.9 GB | 4.63 | 4-bit importance matrix, extra small |
| Q3_K_M | 4.4 GB | 4.12 | 3-bit K-medium |
| IQ3_M | 4.2 GB | 3.94 | 3-bit importance matrix, medium |
| Q3_K_S | 4.0 GB | 3.80 | 3-bit K-small |
| IQ3_XXS | 3.7 GB | 3.51 | 3-bit importance matrix, extra-extra small |
| Q2_K | 3.6 GB | 3.41 | 2-bit K-quant, smallest size |
Q4_K_M or Q5_K_M — excellent quality-to-size ratioQ6_K or Q8_0IQ4_XS or Q3_K_MQ2_KCompatible with llama.cpp, LM Studio, Ollama, text-generation-webui, and any GGUF-compatible inference engine.
llama-cli -m Qwen3.5-9B-GLM5.1-Distill-v1-Q4_K_M.gguf -p "Hello, world!"
Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1 — a Qwen3.5-9B variant distilled with GLM5.1.