Downloads · 30 days
60
27% of all-time downloads
neopolita/Qwen3.6-19B-A3B-Niwaki-v2-4bit-mlx
Qwen3.6-19B-A3B-Niwaki-v2-4bit-mlx is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
Downloads · 30 days
60
27% of all-time downloads
All-time downloads
220
Public
Parameters
1.6K
11.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors11.7 GB · 100%
How the weights are stored.
U3218.6B · 100%
From the Hugging Face model README

Qwen3.6-35B-A3B pruned to 19B total / ~3.3B active
parameters, stored at 4-bit — 3.8× less expert memory than
the 8-bit reference. Loads with the 30-line shim shipped in this repo;
everything else is stock mlx_lm.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts.
A paper with the full method and measurements is coming soon.
Full evaluation protocol: WikiText-2 (145 × 2048-token windows) and C4 (256 × 2048-token windows) perplexity; task average over arc_easy, hellaswag, piqa, winogrande, and boolq, paired against the reference.
| model | expert storage | wt2 ppl ↓ | task avg ↑ | retention | generation d2 (avg/min) |
|---|---|---|---|---|---|
| reference (8-bit) | 34.2 GB | 6.76 | 0.790 | 100% | 0.63 / 0.51 |
| Qwen3.6-27B-A3B-Niwaki-2bit-mlx | 7.6 GB (0.221×) | 9.89 | 0.739 | 93.5% | — |
| Qwen3.6-19B-A3B-Niwaki-2bit-mlx | 4.7 GB (0.147×) | 12.38 | 0.712 | 90.1% | — |
| this model | 9.1 GB (0.265×) | 9.42 | 0.737 | 93.3% | 0.80 / 0.56 |
The quality tier of the v2 family: against the same-size first-generation model it trades bytes for a large quality step; against the first-generation flagship (27B) it reaches lower perplexity at a modestly larger byte budget with an equal task average. Generation quality is measured, not assumed: an 8-prompt battery (code, reasoning, chat, creative; 600-token sampled generations) scored by bigram diversity, reference measured under the identical battery.
Task average: five benchmarks from lm-evaluation-harness scored by
per-choice log-likelihood — arc_easy, hellaswag, and piqa report normalized
accuracy (acc_norm); winogrande and boolq report accuracy (acc); the
average is the unweighted mean of the five. Every row is evaluated on the
same 500 samples per task (paired screens), so the numbers are directly
comparable down the column. Perplexity rows use the full window protocol
above, identical windows for every row, loss in fp32, no chat template.
| total parameters | ~19B (from 35B) |
| active parameters per token | ~3.3B (from ~3.8B) |
| layers / routed experts / top-k | 40 / 256 / 8 (layers 10–29 are shared-expert-only) |
| expert intermediate size | 512 (unchanged) |
| expert precision | 4-bit (group 64) |
| backbone precision | 8-bit (untouched) |
| whole-model size on disk | ~11.7 GB |
| runs comfortably in | 24 GB unified memory |
This checkpoint stores no tensors for the removed expert banks, which stock
mlx-lm cannot express — load through the shim in this repo (it stubs the
routed path of the shared-only layers, then defers to mlx_lm.load):
# curl -O https://huggingface.co/neopolita/Qwen3.6-19B-A3B-Niwaki-v2-4bit-mlx/resolve/main/niwaki_v2_load.py
from niwaki_v2_load import load
from mlx_lm import generate
model, tokenizer = load("neopolita/Qwen3.6-19B-A3B-Niwaki-v2-4bit-mlx")
print(generate(model, tokenizer, prompt="...", max_tokens=256))
GGUF builds for llama.cpp: Qwen3.6-19B-A3B-Niwaki-v2-4bit-gguf.
Base model by the Qwen team (Apache 2.0); 8-bit MLX conversion by mlx-community; compression and distillation by the Niwaki project, 2026-08.