Downloads · 30 days
5.4K
49% of all-time downloads
scima/Qwen3.8-Whittle-MoE-27B-A17.8B-GGUF-all-quants
Qwen3.8-Whittle-MoE-27B-A17.8B-GGUF-all-quants is a text generation model from scima. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
(note i have not tested any quantizations that are Q4 or above, let me know if one of the files don't work)
Downloads · 30 days
5.4K
49% of all-time downloads
All-time downloads
11.1K
Public
Repo size
185 GB
Likes
6
Trending 1
Click a slice to open those files.
.gguf185 GB · 100%
From the Hugging Face model README
| file | size | use |
|---|---|---|
Whittle-MoE-27B-A18B-v2.1-Q2_K.gguf | 11.3 GB | smallest usable file |
Whittle-MoE-27B-A18B-v2.1-Q3_K_S.gguf | 12.7 GB | |
Whittle-MoE-27B-A18B-v2.1-Q3_K_M.gguf | 13.9 GB | likely best quality possible for 16GB VRAM, if not Q3_K_S |
Whittle-MoE-27B-A18B-v2.1-Q3_K_L.gguf | 14.7 GB | |
Whittle-MoE-27B-A18B-v2.1-Q4_K_S.gguf | 16.1 GB | |
Whittle-MoE-27B-A18B-v2.1-Q4_K_M.gguf | 17.4 GB | recommended; fits 24 GB VRAM, splits across two 12 GB cards |
Whittle-MoE-27B-A18B-v2.1-Q5_K_S.gguf | 19.0 GB | try using if spare headroom is available |
Whittle-MoE-27B-A18B-v2.1-Q5_K_M.gguf | 19.9 GB | |
Whittle-MoE-27B-A18B-v2.1-Q6_K.gguf | 23.1 GB | |
Whittle-MoE-27B-A18B-v2.1-Q8_0.gguf | 28.7 GB | higher quality reference |
Whittle-MoE-27B-A18B-v2.1-F16.gguf | 53.9 GB | use for creating future quantizations |
(note i have not tested any quantizations that are Q4 or above, let me know if one of the files don't work)
Every donation goes directly to GPU hours, and every GPU hour gets reported, including the failures: ko-fi.com/davida81328
Quantized builds of Whittle MoE 27B (A18B), the post hoc mixture of experts carved from Qwen3.8-27B and taught when to stop talking. v2.1 passed every release bar: 8 percent single turn loop rate (from 69 at first release), 22 percent structured (from 75), zero truncated answers, knowledge battery 28 of 39. All measurements, method, and failure history live on the main model card.
| file | size | use |
|---|---|---|
Whittle-MoE-27B-A18B-v2.1-Q4_K_M.gguf | 17.4 GB | recommended; fits 24 GB VRAM, splits across two 12 GB cards |
Whittle-MoE-27B-A18B-v2.1-Q8_0.gguf | 28.7 GB | higher quality reference |
llama-server -m Whittle-MoE-27B-A18B-v2.1-Q4_K_M.gguf -ngl 99 -c 8192 -fa on --jinja
Any recent llama.cpp build with Qwen3.5 MoE support works. No fork, no patches.