Downloads · 30 days
63
1% of all-time downloads
Fredred89/Qwen3.5-27B-GLM5.1-Distill-v1-GGUF
Qwen3.5-27B-GLM5.1-Distill-v1-GGUF is a machine learning model from Fredred89. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Architecture: DENSE (NOT MoE) — All 27B parameters are active on every token. No expert routing.
Downloads · 30 days
63
1% of all-time downloads
All-time downloads
6K
Public
Repo size
423 GB
Likes
0
Public
Click a slice to open those files.
.gguf182 GB · 100%
From the Hugging Face model README
Architecture: DENSE (NOT MoE) — All 27B parameters are active on every token. No expert routing.
| Property | Value |
|---|---|
| Base model | Qwen3.5-27B-GLM5.1-Distill-v1 |
| Parameters | 27B (all active, dense) |
| Layers | 64 (48 GDN recurrent + 16 full-attention, every 4th layer) |
| Architecture | Qwen3_5ForConditionalGeneration (hybrid GDN) |
| Vocab | 248,320 |
| Context | 262,144 |
| File | Type | Size | PPL | Notes |
|---|---|---|---|---|
...-APEX-Quality-v5.gguf | APEX Q4_K_M + edge upgrades | 16.18 GB | 5.5596 | Best quality — beats built-in Q4_K_M |
...-Q4_K_M.gguf | Built-in Q4_K_M | 15.41 GB | 5.5687 | Baseline |
...-Q8_0.gguf | Q8_0 | ~27 GB | — | High quality reference |
...-F16-fixed2.gguf | F16 | 53.8 GB | ~5.55 | Full precision (fixed block_count) |
Benchmark: wiki.test.raw, c=2048, chunks=10
This uses an APEX-inspired minimal-override strategy adapted for dense models.
What APEX Quality v5 does:
Why this works for dense models:
Key difference from APEX paper: The original APEX paper targets MoE models (Qwen3.5-35B-A3B with 256 routed experts). Its biggest innovation — compressing inactive experts aggressively — doesn't apply here. We only use the layer-gradient principle.
# 1. Convert to F16 and fix metadata (block_count bug: 65 → 64)
python3 convert_hf_to_gguf.py safetensors_source/ --outfile model-F16.gguf --outtype f16
llama-quantize \
--override-kv 'qwen35.block_count=int:64' \
--override-kv 'qwen35.nextn_predict_layers=int:0' \
model-F16.gguf model-F16-fixed.gguf COPY
# 2. Quantize with APEX tensor-type-file (see APEX-Quality-v5.tensor_types.txt)
llama-quantize \
--tensor-type-file APEX-Quality-v5.tensor_types.txt \
model-F16-fixed.gguf APEX-Quality-v5.gguf Q4_K_M
# 3. Benchmark (MUST use c=2048, not default c=512)
llama-perplexity -m APEX-Quality-v5.gguf -f wiki.test.raw -c 2048 --chunks 10 -t 4