Kanana-2-30B-A3B-Instruct-JANG_4M

<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M

A Korean-first JANG_4M MLX bundle of Kakao's Kanana-2 30B-A3B Mixture-of-Experts model, tuned to run fast on Apple Silicon via Osaurus.

Converted from kakaocorp/kanana-2-30b-a3b-instruct-2601.

Why this model

Kanana-2 is one of the strongest open Korean LLMs — Kakao trained it with a tokenizer ~30% more efficient on Korean and a 6-language mix (Korean, English, Japanese, Chinese, Thai, Vietnamese). It is a 30.7B-parameter MoE that activates only ~3B parameters per token (top-6 of 128 routed experts + 2 shared), so you get large-model Korean quality at small-model decode speed. JANG_4M compresses it to ~17 GB so it fits comfortably in unified memory on a MacBook.

Bundle

Quantization policy (JANG_4M)

JANG_4M is a mixed-precision affine scheme designed for MoE: the precision-sensitive pieces stay wide while the redundant expert bulk is compressed.

• 8-bit affine: self-attention projections (MLA q/kv compression + decompression, o_proj).
• fp16 passthrough: MoE router/gate weights (routing is extremely precision-sensitive), all RMSNorms.
• 4-bit affine: routed-expert and dense-MLP gate/up/down projections (the bulk of the weights).
• High-precision bookends: token embeddings and lm_head.

The per-tensor bit widths are recoverable directly from the weight/scale shapes; jang_config.json is authoritative for the JANG runtime.

Modalities

Languages

Korean (primary), English, Japanese, Chinese, Thai, Vietnamese.

Reasoning & tools

The source chat template is ChatML-style (<|im_start|>role) with <think>…</think> reasoning and <tool_call>{json}</tool_call> tool calls. The bundle stamps reasoning_parser: qwen3, tool_parser: qwen. Thinking is off by default for the instruct variant; enable it through your runtime's thinking toggle.

Runtime

Built for the Osaurus / vMLX Apple-Silicon runtime. Loading requires a deepseek_v3-aware MLX loader that honors jang_config.json and the bundle's mixed per-tensor bit widths. tokenizer_config.json, generation_config.json, and chat_template.jinja are preserved from the source.

Notes

• This is a freshly converted bundle. The base model's published Korean benchmarks (e.g. KMMLU) reflect the unquantized weights; JANG_4M-level evaluation for this bundle is pending and will be added here when complete.
• MTP: the source ships an MTP layer; speculative decoding is not enabled in this bundle (mtp: none for autoregressive decode).

Credits

• Base model: Kakao Corp. — Kanana-2, released under the Kanana License.
• JANG conversion & Osaurus packaging: Jinho Jang — [email protected]