Downloads · 30 days
229
10% of all-time downloads
neopolita/Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-gguf
Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-gguf is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as other.
Downloads · 30 days
229
10% of all-time downloads
All-time downloads
2.4K
Public
Repo size
112 GB
Likes
4
Public
Click a slice to open those files.
.gguf112 GB · 100%
From the Hugging Face model README

GGUF builds of Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-mlx — Qwen3.8-Flash-Next pruned to 113B total / ~5.4B active parameters (routed experts kept on 32 of the 48 layers) — for llama.cpp on CUDA, Vulkan, Metal and CPU. Requires the neopolita-llama.cpp fork (see Usage): stock llama.cpp cannot represent this model's layout yet.
Niwaki (庭木): routed experts pruned using statistics of their routed tokens over a mixed web/code/chat/reasoning corpus, reconstructed to compensate, then briefly distilled from the full model, and stored at low precision. A paper with the full method is coming soon.
| file | size | wt2 ppl (llama.cpp, 512-ctx) |
|---|---|---|
| Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-UD-Q3K.gguf (recommended) | 49.1 GB | 7.90 ±0.05 |
| Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-UD-Q4K.gguf | 62.8 GB | 7.70 ±0.05 |
Protocol: llama-perplexity, WikiText-2 test, 512-token windows. These llama.cpp numbers are not directly comparable to the MLX repo's 2048-window benchmarks; the canonical measurements (against the reference model) live there.
Both builds are quantized structure-aware, mirroring the MLX artifact's native allocation: the always-active backbone (attention, linear attention, hyper-connections, shared experts) at Q6_K, embeddings and output at Q8_0, the pruned routed experts on a compact carrier (Q3_K, or Q4_K in the UD-Q4K file), the n-gram embedding table at Q2_K (Q3_K in UD-Q4K), and the distilled output maps at Q8_0.
| total / active parameters | 113B / ~5.4B |
| layers / routed experts / top-k | 48 / 512 / 10 (routed experts on 32 of the 48 layers) |
| expert intermediate size | 512 (448 used; zero-padded for the quantization block size) |
| context | as base model |
| modality | text only (vision tower not included; MTP draft head not included) |
git clone https://github.com/Neopolita/neopolita-llama.cpp
cd neopolita-llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release # add -DGGML_METAL=ON / -DGGML_CUDA=ON / -DGGML_VULKAN=ON as usual
cmake --build build -j --target llama-cli llama-server llama-completion
hf download neopolita/Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-gguf Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-UD-Q3K.gguf --local-dir .
./build/bin/llama-cli -m Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-UD-Q3K.gguf
# or serve:
./build/bin/llama-server -m Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-UD-Q3K.gguf
On an M4 Max (Metal) the UD-Q3K file decodes at ~35 tokens/s.
The fork adds three small, general features to llama.cpp's qwen4exp
architecture, needed to store this model at its real size:
ple.row_pack),
so block-256 quantization types apply to its 160-wide rows; otherwise the
table alone is 28.8 GB at 4.5 bits;ffn_out_map), so the distilled healing
maps ship as tensors instead of being folded into 3-bit weights.Files produced by the fork's converter load only in the fork; upstreaming is planned.
Base model by the Qwen team (see license); pruning, distillation and GGUF builds by the Niwaki project, 2026-09.