Spark-X2.5-4B-JANG_6M

<p align="center"> <a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a> </p>

OsaurusAI/Spark-X2.5-4B-JANG_6M

Spark-X2.5-4B at 3.41 GiB — the size-first bundle. Fastest of the pair, and the one to run if you want headroom.

A JANG bundle of XHToken/Spark-X2.5-4B — a 4.112B dense reasoning + tool-use model — quantized for Apple Silicon / MLX. Attention and the tied embedding stay at 8-bit; the MLP carries the 6-bit budget. Every group scale is bfloat16, and the per-head attention gate and all norms are kept in full precision.

Sibling bundle: JANGQ-AI/Spark-X2.5-4B-JANG_8M — higher fidelity at 4.07 GiB.

Quality — measured, not estimated

15,839 teacher-forced positions on held-out prompts, versus the bf16 source. Calibration and evaluation prompts come from disjoint corpora, asserted at eval time.

Decode measured as the median of 4 probes at a fixed condition (512-token prompt, 128 generated), first probe discarded, on an M5 Max. Prefill 5046 tok/s.

There is no stock-MLX comparison row, and that is not an omission. mlx-lm has no spark2_5 architecture — not in any release, not on main — so no stock MLX quantization of this model can be loaded or scored. The MLX runtime ships with these bundles.

Calibration

1,759,492 calibration tokens spanning code, agentic/tool-call transcripts, academic multiple-choice, general chat, Chinese, long-context, science and security prompts. One capture drives all three of activation-aware scaling, per-channel importance, and the error-corrected weight fit; every one of the 36 layers is covered.

Running it in Osaurus

Built for the Osaurus harness. Point Osaurus at the repo and it serves with the sampling, reasoning and tool-call contract below already declared in the bundle — jang_config.json carries the parser names, the dialect and the delimiters, so the harness wires itself with no per-model configuration.

spark2_5 is a new architecture and is not in released mlx-lm, so a bare mlx_lm.load will not resolve it — the runtime ships with the harness.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("OsaurusAI/Spark-X2.5-4B-JANG_6M")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is 84 * 3 / 2?"}],
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=-1 (-1 = no truncation). Vendor defaults; declared identically in generation_config.json and jang_config.json.
• Thinking is ON by default. The chat template opens the rail in the prompt: the generation prompt ends with <|Bot|><think>. With chat_template_kwargs={"enable_thinking": false} it ends <|Bot|></think> instead. A reasoning parser that waits for a literal <think> in the output stream will never see one — use reasoning_parser: qwen3, declared in the bundle.
• Tools render into the first system block (## Tools + a <tools> JSON list), so adding or removing a tool rewrites the prompt prefix and forces a full re-prefill. Calls come back as <tool_call>NAME<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> with no newlines; tool_parser: spark25 is declared in the bundle.
• EOS 1 · BOS 0 · context 131,072 (1M positions native) · text only.
• generation_config.json carries upstream's non-standard max_tokens key verbatim and has no max_new_tokens. Pass your own generation cap.

What's in the bundle

• 181 quantized tensors, each with an explicit per-module quantization entry in config.json — nothing falls back to a top-level default.
• 109 full-precision passthrough tensors: the per-head attention gate and all norms.
• bfloat16 group scales throughout.
• Zero misaligned tensor payloads (652/652 verified), so no realigning copy at load.
• No auto_map and no remote-code .py files — the runtime comes from the harness, so loading never stops on a trust_remote_code prompt.

Quantized and verified by Jinho Jang — [email protected]. Base model © XHToken, Apache-2.0 (inherited).