Kimi-K3-REAP640ja-IQ1_S-GGUF

Kimi-K3-REAP640ja-IQ1_S-GGUF

The Japanese-calibrated sibling of Kimi-K3-REAP640-IQ1_S-GGUF: Kimi-K3 for Japanese, on a single 512 GB Mac Studio.

Same recipe as REAP640 — Unsloth's UD-IQ1_S dynamic 1-bit quant (594 GB, all 896 experts) REAP-pruned to 640 experts / 441 GB — but the keep-list comes from a Japanese + Chinese calibration instead of English + code. Same size, same expert count per layer; the two builds differ only in which 640 experts survive (~473 of 640 per layer are shared, ~167 swapped).

On ELYZA-tasks-100 (Japanese instruction benchmark, blinded LLM judge) this build scores 4.16/5 where the en+code sibling scores 1.81/5, winning the blinded pairwise comparison 83–6 with 11 ties. Japanese held-out perplexity drops from 19.46 to 4.46.

Chinese rides along (it was 10% of the calibration and Japanese borrows heavily from Chinese experts — 42.8% top-expert overlap): zh perplexity is 4.10 vs the sibling's 7.93.

This is not a coding build. Code perplexity roughly doubles (2.00 → 3.87) and agentic use is untested here. For coding agents, use REAP640.

Full write-up (Japanese): 枝刈りKimi K3の日本語版を作って、ELYZA-tasks-100で比べた

Provenance: the exact pruning plan is published — plans/reap_plan_640ja.json (sha256 30b862a465790480…, full hash in SHA256SUMS); plan + source quant + the MIT slicer reproduce this file's bytes. Split-half: calibrating on Japanese alone vs Chinese alone agrees on 83.8% of the keep-set (oracle retention 89–91%) — see plans/README.md.

Verified vs. not verified

Honest scorecard: exactly what has been measured, and what has not.

Verified:

Verified operating envelope — what this card's numbers actually cover:

Agentic tool-calling sessions have been run only on the en+code sibling — its SWE-Lancer numbers belong to that build alone. The code-perplexity doubling above is the honest predictor for coding work here, and the judge flagged factual slips inside fluent Japanese: 1.6-bit experts are fluent before they are precise.

Download & run

This repo ships one 441 GB GGUF, no shards (the Hub allows files up to 500 GB; hf download resumes interrupted transfers):

hf download hellohazime/Kimi-K3-REAP640ja-IQ1_S-GGUF \
    Kimi-K3-REAP640ja-IQ1_S.gguf --local-dir .

Kimi-K3 support is not in mainline llama.cpp yet. Build the Unsloth fork at its K3 PR (built on top of llama.cpp PR #26185):

git clone https://github.com/unslothai/llama.cpp
cd llama.cpp && git fetch origin pull/48/head:kimi-k3 && git checkout kimi-k3
cmake -B build -DGGML_METAL=ON        # Apple Silicon; use -DGGML_CUDA=ON on NVIDIA
cmake --build build --config Release -j --target llama-server

./build/bin/llama-server -m Kimi-K3-REAP640ja-IQ1_S.gguf \
    --port 8090 -ngl 99 -c 131072 --jinja --cache-reuse 0 \
    --temp 1.0 --top-p 0.95

• --cache-reuse 0 is required: partial prefix-cache reuse corrupts the KDA recurrent state (known issue, see the PR discussion).
• K3 is thinking-only; reasoning arrives in reasoning_content. Control depth with chat_template_kwargs: {"thinking_effort": "low" | "high" | "max"}.
• Sampling per Moonshot: temperature 1.0, top_p 0.95.

How it was made

Identical pipeline to the sibling build, documented in 01554/kimi-k3-gguf-prune. A tagged multi-domain calibration corpus (ja 30% / en 25% / code 25% / zh 10% / 5-language tail 10%, token shares) was streamed once through the 1.56 TB MXFP4 source with per-source saliency recording; the keep-640 plan sums only the lang-ja + chinese labels. Saliency measurement and planning use pipenetwork's kimi-k3-mlx scripts (reap_calibrate.py / reap_subset.py / reap_plan.py); the GGUF slicing (byte-slab copy along the expert axis, router rows and exp_probs_b renumbered to keep order) is this project's only original code.

Credits: Moonshot AI (Kimi-K3), Unsloth (dynamic 1-bit quant whose protected router/norms this build inherits), Cerebras REAP (saliency criterion), kimi-k3-mlx (calibration machinery), ELYZA (ELYZA-tasks-100).

日本語の説明

Moonshot AIの2.8兆パラメータモデル Kimi-K3を、Mac Studio(512GB) 1台で動くサイズ(441GB)に枝刈りした日本語版です。

公開済みのREAP640は 英語+コードで校正したため、日本語を入れると中国語混じりの出力に崩れます。 この版は校正を日本語+中国語に変えて、同じ640個構成で作り直したものです。 サイズも構成も同じで、残っているexpertの顔ぶれだけが違います。

ELYZA-tasks-100(盲検・提示順ランダムのClaude Sonnet判定)で4.16/5。 英語+コード版は同条件で1.81/5でした。判定者がリーダーボード(GPT-4系)と 違うため、数値を外部のスコアと直接比較はできません。

注意点は2つ。コーディング能力は劣化しています(code perplexity 2.00→3.87)。 エージェント用途の検証(Kimi Code CLI、SWE-Lancer)はこの版では行っていません。 その用途にはREAP640を使ってください。

作った経緯と実測の詳細: 枝刈りKimi K3の日本語版を作って、ELYZA-tasks-100で比べた