Downloads · 30 days
18.3K
14% of all-time downloads
sh0wie/Qwen3.8-Flash-Next-REAP-288-GGUF
Qwen3.8-Flash-Next-REAP-288-GGUF is a text generation model from sh0wie. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as other.
This is the REAP-288 build in GGUF, for llama.cpp and the tools built on it (Ollama, LM Studio, and similar). It is the 180B-class MoE with its experts pruned 512 - 288 per layer, converted directly from the bf16 sour…
Downloads · 30 days
18.3K
14% of all-time downloads
All-time downloads
131K
Public
Repo size
302 GB
Likes
12
Trending 1
Click a slice to open those files.
.gguf302 GB · 100%
From the Hugging Face model README
This is the REAP-288 build in GGUF, for llama.cpp and the tools built on it (Ollama, LM Studio, and similar). It is the 180B-class MoE with its experts pruned 512 -> 288 per layer, converted directly from the bf16 source in a single quantization step, so each file loses accuracy exactly once rather than stacking a re-quantization on top of an existing 4-bit build.
The pruned model scores 91.5% on HumanEval (149 of 164), against 93.9% for the unpruned full model. That number is measured on the MLX 4-bit lineage; the GGUF quants share the same kept-expert selection and derive from the same bf16 source.
If you run MLX on Apple Silicon instead of llama.cpp, use the 4-bit MLX build. The pmlx engine and its speedups apply to the MLX builds, not to GGUF; nothing on this card depends on it.
Converted from the bf16 source. Pick one quant; larger files are higher
fidelity and slower. Take Q4_K_M for the smallest, fastest build, Q8_0 for
the highest quality, and Q5_K_M if you want a middle point.
| Quant | Size | Notes |
|---|---|---|
Q4_K_M | 78 GB | good default: smallest and fastest, strong quality |
Q5_K_M | 87 GB | higher fidelity, more memory |
Q8_0 | 116 GB | near-lossless, largest and highest quality |
llama.cpp:
# build llama.cpp, then:
llama-server -m Qwen3.8-Flash-Next-REAP-288-Q4_K_M.gguf --port 8080
# or a one-shot completion:
llama-cli -m Qwen3.8-Flash-Next-REAP-288-Q4_K_M.gguf -p "Refactor this function to add input validation."
Ollama (pulls the quant straight from this repo):
ollama run hf.co/sh0wie/Qwen3.8-Flash-Next-REAP-288-GGUF:Q4_K_M
Qwen/Qwen3.8-Flash-Next: upstream weights.Qwen Community License 1.0, inherited from the base model Qwen/Qwen3.8-Flash-Next.