Downloads · 30 days
74
16% of all-time downloads
neopolita/Qwen3.8-84B-A53B-Niwaki-v2-experimental-mlx
Qwen3.8-84B-A53B-Niwaki-v2-experimental-mlx is a text generation model from neopolita. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as other.
Downloads · 30 days
74
16% of all-time downloads
All-time downloads
451
Public
Parameters
84B
63 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors63 GB · 100%
How the weights are stored.
U3284B · 100%
From the Hugging Face model README

⚠️ EXPERIMENTAL RESEARCH ARTIFACT — NOT FOR REAL USE. This model is a measurement instrument, not a usable assistant. It is Qwen3.8-2.4T-A95B with 98% of its routed experts deleted (2.4T → 84B total / ~53B active parameters, 63 GB on disk) and a short distillation heal. It generates fluent, well-structured text that is frequently and confidently wrong — knowledge that lived in the deleted experts is gone, and no sampling setting brings it back. Published so the curious can poke at what survives extreme MoE pruning at frontier scale.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts — here pruned far past the point of use, to see where the form gives way.
A paper with the full method and measurements is coming soon.
The 2.4T reference routes each token through 10 of 512 experts per layer across 92 layers. This artifact keeps: the full attention/backbone (8-bit), every shared expert (healed, 8-bit), the original routers, and compact 2-bit banks holding only the top-96 most-routed experts in the first 6 layers and top-40 in the last 3 — 664 of 47,104 routed experts (1.4%). Routing renormalizes exactly over the kept experts. A ~0.5M-token knowledge-distillation heal (teacher logits from the reference model) recovered about half of the deletion damage in log-perplexity before plateauing.
Screen protocol: WikiText-2 and C4 perplexity over 64 × 1024-token windows each (fp32 loss, no chat template) — smaller than the full protocol of the main Niwaki releases; every row below uses the identical windows.
| model | disk | wt2 ppl ↓ | c4 ppl ↓ |
|---|---|---|---|
| Qwen3.8-2.4T-A95B (teacher, API anchor) | ~4,890 GB (bf16) | 2.77* | — |
| shared-experts-only strip (no routed experts) | 52.1 GB | 46.21 | 76.90 |
| + protected expert banks (unhealed) | 63.0 GB | 39.87 | 64.04 |
| this model (protected + healed, folded) | 63.0 GB | 23.13 | 29.95 |
* teacher anchor measured via API echo-scoring on the same text, its own tokenization; indicative, not window-identical. Task-suite evaluation was not run for this artifact.
Form survives; knowledge does not. The model plans, formats code blocks, self-corrects, and holds discourse structure — then confabulates the substance (asked for Tetris, it confidently invents a different game). Ranking-style capabilities recover much faster than open generation. Expect coherent wrongness, repetition under greedy decoding, and no factual reliability whatsoever.
Loads with the same 30-line shim as the other Niwaki releases
(niwaki_v2_load.py, shipped in this repo); everything else is stock
mlx_lm. Needs ~70 GB free unified memory; ~30 tok/s on an M4 Max.
from niwaki_v2_load import load
import mlx_lm
from mlx_lm.sample_utils import make_sampler, make_logits_processors
model, tok = load("neopolita/Qwen3.8-84B-A53B-Niwaki-v2-experimental-mlx")
out = mlx_lm.generate(
model, tok,
prompt=tok.apply_chat_template([{"role": "user", "content": "Hello!"}],
tokenize=False, add_generation_prompt=True),
max_tokens=256,
sampler=make_sampler(temp=1.0, top_p=0.95, top_k=20),
logits_processors=make_logits_processors(repetition_penalty=1.15,
repetition_context_size=256),
)
Shipped sampling defaults (temp 1.0, top-k 20, top-p 0.95) follow the base model; the repetition penalty is recommended — the healed model loops without it.
92 layers (hybrid linear-attention + gated attention), hidden 8192, 512 routed experts top-10 + 1 shared expert per MoE layer in the reference; here 83 layers are shared-expert-only and 9 layers carry kept-expert banks (96×6 early, 40×3 late, 2-bit g64). Healed output maps are folded into the down-projections (zero extra parameters). Storage: 8-bit backbone + 2-bit banks, mixed precision, 63.0 GB total.
Not documented here yet — a paper with the full method and measurements is coming soon.