Downloads · 30 days
31
7% of all-time downloads
neopolita/Qwen3.6-19B-A3B-Niwaki-2bit-mlx
Qwen3.6-19B-A3B-Niwaki-2bit-mlx is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
Downloads · 30 days
31
7% of all-time downloads
All-time downloads
424
Public
Parameters
1.8K
7.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7.6 GB · 100%
How the weights are stored.
U3218.6B · 100%
From the Hugging Face model README

Qwen3.6-35B-A3B pruned to 19B total / 3.3B active
parameters, stored at 2-bit — 6.8× less expert memory than
the 8-bit reference. Stock mlx_lm, no custom code.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts: every routed expert is individually width-pruned using statistics of its routed tokens over a mixed web/code/chat/reasoning corpus, reconstructed to compensate, then distilled from the full model on the same mixed corpus, and stored at low precision. The result is an ordinary MoE — smaller, same architecture, standard fused kernels.
A paper with the full method and measurements is coming soon.
Full evaluation protocol: WikiText-2 perplexity over 256 × 2048-token windows; task average over the complete arc_easy, hellaswag, piqa, winogrande, and boolq test sets, paired against the reference.
| model | expert storage | wt2 ppl ↓ | task avg ↑ | retention | generation d2 (avg/min) |
|---|---|---|---|---|---|
| reference (8-bit) | 34.2 GB | 6.76 | 0.796 | 100% | 0.89 / 0.77 |
| this model | 5.0 GB (0.147×) | 12.38 | 0.742 | 93.3% | 0.89 / 0.75 |
Generation quality is measured, not assumed: an 8-prompt battery (code, reasoning, chat, creative; 600-token sampled generations) scored by bigram diversity.
| total parameters | 19B (from 35B) |
| active parameters per token | ~3.3B (from ~3.8B: 2.8B backbone + 8 pruned experts) |
| layers / routed experts / top-k | 40 / 256 / 8 |
| expert intermediate size | 256 (from 512) |
| expert precision | 2-bit (group 64) |
| backbone precision | 8-bit (untouched) |
| whole-model size on disk | ~8.0 GB |
| runs comfortably in | 12–16 GB unified memory |
from mlx_lm import load, generate
model, tokenizer = load("neopolita/Qwen3.6-19B-A3B-Niwaki-2bit-mlx")
print(generate(model, tokenizer, prompt="...", max_tokens=256))
GGUF builds for llama.cpp: Qwen3.6-19B-A3B-Niwaki-2bit-GGUF (UD-Q3K recommended + Q4_K_M).
Base model by the Qwen team (Apache 2.0); 8-bit MLX conversion by mlx-community; pruning and distillation by the Niwaki project, 2026-08. Method details: paper coming soon.