Downloads · 30 days
326
14% of all-time downloads
neopolita/Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-gguf
Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-gguf is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as other.
Downloads · 30 days
326
14% of all-time downloads
All-time downloads
2.3K
Public
Repo size
115 GB
Likes
4
Public
Click a slice to open those files.
.gguf115 GB · 100%
From the Hugging Face model README

GGUF builds of Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-mlx — Qwen3.8-Flash-Next pruned to 119B total / ~5.6B active parameters, with the program-state behaviours we test (variable tracking, two-step arithmetic, multi-hop lookups) at the unpruned model's level — for llama.cpp on CUDA, Vulkan, Metal and CPU. Requires the neopolita-llama.cpp fork (see Usage): stock llama.cpp cannot represent this model's layout yet.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts. A paper with the full method and measurements is coming soon.
Project page, with the releases and their measurements side by side: niwakiai.com
| file | size | wt2 ppl (llama.cpp, 512-ctx) |
|---|---|---|
| Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-UD-Q3K.gguf (recommended) | 51.0 GB | 7.04 ±0.05 |
| Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-UD-Q4K.gguf | 64.3 GB | 6.88 ±0.04 |
The first generation's GGUF builds under the identical protocol:
| model (UD-Q3K builds) | size | wt2 ppl |
|---|---|---|
| Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit | 49.1 GB | 7.90 ±0.05 |
| Qwen3.8-Flash-Next-99B-A5B-Niwaki-3bit | 42.5 GB | 8.79 ±0.06 |
This build reads 11% lower perplexity than the first generation's 113B build for 1.9 GB more (the UD-Q4K file: 6.88 against 7.70).
Protocol: llama-perplexity, WikiText-2 test, 512-token windows. These llama.cpp numbers are not directly comparable to the MLX repo's 2048-window benchmarks; the canonical measurements (against the reference model: task suite, KL divergence, program-state behaviours, generation battery) live there.
Program-state check through llama.cpp (UD-Q3K file, the suite's 300
variable-tracking items, greedy decoding through llama-server): tracking
1.00 / 0.96, tracking with distractor statements 0.93 — the MLX
artifact scores 0.98 / 0.95 / 0.93 on the same items.
Both builds are quantized structure-aware, mirroring the MLX artifact's native allocation: the always-active backbone (attention, linear attention, hyper-connections, shared experts) at Q6_K (Q8_0 where a row is narrower than a 256-block), embeddings and output at Q8_0, the routed experts on a compact carrier — gate/up projections Q3_K and down projections IQ4_NL (Q4_K and Q5_0 in the UD-Q4K file; the 640-wide down rows take a 32-block type) — and the n-gram embedding table at Q2_K (Q3_K in UD-Q4K).
Speed (M4 Max, Metal, llama-bench, 512-token prompt / 128 generated): UD-Q3K 557 tok/s prompt, 30 tok/s decode; UD-Q4K 561 / 31 tok/s.
| total / active parameters | 119B / ~5.6B |
| layers / routed experts / top-k | 48 / 512 / 10 (24 layers keep all 512 experts; 24 layers keep 16 each, and their routed path contributes only when the router picks one of them) |
| expert intermediate size | 640 (unchanged) |
| context | as base model |
| modality | text only (vision tower not included; MTP draft head not included) |
git clone https://github.com/Neopolita/neopolita-llama.cpp
cd neopolita-llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release # add -DGGML_METAL=ON / -DGGML_CUDA=ON / -DGGML_VULKAN=ON as usual
cmake --build build -j --target llama-cli llama-server llama-completion
hf download neopolita/Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-gguf Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-UD-Q3K.gguf --local-dir .
./build/bin/llama-cli -m Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-UD-Q3K.gguf
# or serve:
./build/bin/llama-server -m Qwen3.8-Flash-Next-119B-A5B-Niwaki-v2.4-3bit-UD-Q3K.gguf
Support for this file layout landed in the fork together with this release:
if you built the fork for the first-generation files, git pull and rebuild.
These files use two small, general additions to llama.cpp's qwen4exp
architecture, needed to store this model at its real size:
ffn_exp_ids): 24 layers keep 16 of their 512 experts. The router still
scores all 512 and the top-10 is taken exactly as in the base model; an
expert that is not stored contributes nothing. Stock llama.cpp needs uniform
expert tensors, which would add ~28 GB of all-zero experts here;ple.row_pack),
so block-256 quantization types apply to its 160-wide rows; otherwise the
table alone is 28.8 GB at 4.5 bits.Files produced by the fork's converter load only in the fork; upstreaming is planned.
Base model by the Qwen team (see license); pruning, recovery training and GGUF builds by the Niwaki project, 2026-09.