Kimi-K3-mlx-reap160-2bit

Kimi-K3 MLX: 160 experts, 2-bit (181 GB)

An MLX build of moonshotai/Kimi-K3 (2.78 T total, 104 B active) pruned to 160 of 896 routed experts per layer and quantized to 2-bit at group 128, so it runs resident on a 192 GB Apple Silicon machine. In short, about half the memory of the smallest previously published MLX tier.

Read this before downloading

This is a code-capable model with degraded factual recall. It is not a general-purpose K3.

Measured with greedy decoding, 40 new tokens:

If you need factual recall or Chinese, use a larger tier. If you want a K3 that writes code on a 192 GB Mac, this is the only build we know of that fits.

Why the factual recall is gone, precisely

A token's MoE output combines its top-k experts with gate weights renormalized to sum to 1, so the model cannot down-weight an expert it dislikes. With p_j = P(expert j is in a token's top-k), a kept set S serves Σ_{j∈S} p_j / top_k of the router's demand. We measured this on a 32k-token calibration set:

At 160 experts a token reaches ~8.3 of the 16 experts it wanted. That is enough for code and not for facts. Recovering full capability needs ~77% retained mass ≈ 376 experts ≈ 366 GB, which does not fit 192 GB at MLX's 2-bit floor. So this limitation is arithmetic, not a tuning failure.

Things we tried that did not help, so you don't repeat them:

• Lower top_k (16 → 12 → 8 → 6): the France prompt loops identically at every value, and Chinese gets worse. The damage is missing knowledge, not misweighted survivors.
• A weight-only criterion (AIMER): only 1.25× better than random. At 137 experts it retained 19.4% and every prompt looped.
• Shared-basis + low-rank factorization (keep all 896 experts as W_base + A_jB_j): closed by measurement — the shared mean alone has 99.9% relative error and rank 256 still 86%. K3's experts do not share a basis.

Usage

# pip install mlx mlx-lm
# this repo bundles kimi_k3.py; register it with mlx_lm first:
#   cp kimi_k3.py $(python -c "import mlx_lm.models,os;print(os.path.dirname(mlx_lm.models.__file__))")/
from mlx_lm import load, generate
model, tokenizer = load("./Kimi-K3-mlx-reap160-2bit")
print(generate(model, tokenizer, "def merge_intervals(intervals):", max_tokens=128))

Raise the Metal wired limit — it defaults to ~75% of RAM, below this build:

sudo sysctl iogpu.wired_limit_mb=185000

What has and has not been checked

The prune plan and the REAP saliency ship with the weights, so the selection is auditable and the retained-mass numbers above are reproducible.

Attribution

• Base model: moonshotai/Kimi-K3.
• Loader and streaming converter vendored from PipeNetwork/kimi-k3-mlx.
• REAP expert saliency: arXiv 2510.13999.
• Prior art at 512 GB: pipenetwork/Kimi-K3-REAP80-MLX-mxfp4-q8.