Downloads · 30 days
318
19% of all-time downloads
neopolita/Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-mlx
Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-mlx is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as other.
Downloads · 30 days
318
19% of all-time downloads
All-time downloads
1.7K
Public
Parameters
86.9B
42.2 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors42.2 GB · 100%
How the weights are stored.
U3286.9B · 100%
From the Hugging Face model README

Qwen3.8-Flash-Next pruned to 113B total / ~5.4B active parameters, stored
at 3-bit — 3.1× less expert memory than the 4-bit reference, 42.2 GB on disk
(13 GB below any 2-bit cast of the unpruned model), for 64 GB Macs; keeps 99.8%
of the reference's task average. Stock mlx-vlm
plus the small loader shipped in this repo.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts: routed experts are pruned using statistics of their routed tokens over a mixed web/code/chat/reasoning corpus, reconstructed to compensate, then distilled from the full model on the same mixed corpus, and stored at low precision. The result is the same architecture with fewer, narrower experts.
A paper with the full method and measurements is coming soon.
Evaluation protocol: WikiText-2 perplexity over 64 × 2048-token windows (fp32 loss, no chat template); task average over arc_easy, hellaswag, piqa, winogrande and boolq (zero-shot, 500 examples each), paired against the reference on the identical examples.
| model | expert storage | wt2 ppl ↓ | task avg ↑ | retention | generation d2 (avg/min) |
|---|---|---|---|---|---|
| reference (4-bit) | 75.5 GB | 5.06 | 0.759 | 100% | 0.78 / 0.62 |
| this model | 24.7 GB (0.327×) | 7.08 | 0.757 | 99.8% | 0.66 / 0.42 |
Generation quality is measured, not assumed: an 8-prompt battery (code,
reasoning, chat, creative; 600-token sampled generations) scored by bigram
diversity. Known limitation (measured): with thinking on, this model reasons at length — it used the whole 600-token budget on seven of the eight prompts where the reference stopped early on four — which lowers its diversity score (0.66 vs 0.78; minimum 0.42 on the haiku prompt); the answers themselves are on target. Use enable_thinking=False for short-form tasks.
| total parameters | 113B (from ~177B) |
| active parameters per token | ~5.4B (from ~6.7B) |
| layers / routed experts / top-k | 48 / 512 / 10 (routed experts kept on 32 of the 48 layers) |
| expert intermediate size | 448 (from 640) |
| expert precision | 3-bit (group 64) |
| n-gram embedding table (PLE) precision | 2-bit (group 128) |
| backbone precision | 4-bit (untouched) |
| whole-model size on disk | 42.2 GB |
| runs comfortably in | 64 GB unified memory |
| modality | text only (vision tower not included) |
Requires mlx-vlm >= 0.7.0rc0 and mlx >= 0.32, plus niwaki_flash_load.py
from this repo (it tells stock mlx-vlm about the artifact's layout).
pip install -U mlx mlx-vlm transformers
hf download neopolita/Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-mlx --local-dir ./niwaki-flash-113b
import mlx_vlm
from niwaki_flash_load import load # niwaki_flash_load.py from this repo
model, processor = load("./niwaki-flash-113b")
messages = [{"role": "user", "content": "Explain hyper-connections in two sentences."}]
text = processor.tokenizer.apply_chat_template(messages, add_generation_prompt=True,
tokenize=False, enable_thinking=False)
print(mlx_vlm.generate(model, processor, text, max_tokens=300, verbose=False).text)
enable_thinking=False skips the reasoning preamble; leave it on for harder
prompts. On an M4 Max: ~31 tokens/s, ~42 GB peak.
GGUF builds for llama.cpp (CUDA, Vulkan, Metal, CPU; they need the neopolita-llama.cpp fork): Qwen3.8-Flash-Next-113B-A5B-Niwaki-3bit-gguf (UD-Q3K 49.1 GB recommended, UD-Q4K 62.8 GB).
Base model by the Qwen team (see license); 4-bit MLX conversion by mlx-community; pruning and distillation by the Niwaki project, 2026-09. Method details: paper coming soon.