Raptor-v0.5-8B-A1B-JANG_6M

<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

OsaurusAI/Raptor-v0.5-8B-A1B-JANG_6M

Superseded by Raptor 0.6. Raptor v0.5 stays available, but the current Raptor is OsaurusAI/Raptor-0.6-4B-JANG_6M, built on a different backbone (Spark-X2.5-4B, dense, spark2_5). None of this card's runtime, parser or cache notes apply to it — read its own card.

The official Raptor v0.5. A calibrated JANG_6M quantization of Ling-3.0-tiny-Osaurus-v2mcp — Ling-3.0-tiny with a light r4/alpha8 LoRA merged to BF16. Base model behavior is intentionally preserved; the tune adds familiarity with the Osaurus Mac harness tools (native schemas, tool routing across Osaurus surfaces, observation-grounded final synthesis, concise single-call economy) and with Osaurus declarative configuration (osaurus_config / osaurus_inspect / osaurus_help): MCP server setup (HTTP/OAuth, stdio with env secret refs, bearer token refs, enable/disable) and onboarding — providers and API keys, models, agents, memory, channels, schedules, watchers, knowledge collections, tool policies.

This build (v2mcp) supersedes the previous v0.5 weights (v47-u4 tune). On the tune's own probe batteries it improves every measured surface over its predecessor: 48-case defect sweep 141/144 (was 138), Osaurus tool-surface probe 72/84 (was 69), segmented interleave chains 14/24 (was 7), declarative-config probe 112/198 (was 70).

Lineage

Architecture

Per-layer decode state for KDA is a fixed [16,128,128] recurrent tensor plus three short-conv buffers — it does not grow with context.

Quantization: JANG_6M (calibrated)

Two calibration captures on the merged weights (not the base), feeding the three calibrated stages:

• AWQ scales fit from a per-channel activation capture taken with the reference HF implementation on the Osaurus calibration corpus (133,886 tokens of harness renders + general text); scales folded into the producing norms with router compensation, fold invariant proven exact (2.5e-16 in f64) before use. All 23 MoE input groups improved.
• Hessian-trace mixed-bit allocation from a fresh 1.36M-token MLX capture, over the 69 routed-expert tensor groups: 6 groups at 8-bit, 51 at 6-bit, 12 at 5-bit — budget-neutral water-filling (achieved exactly 6.0000 bits/weight) driven by measured sensitivity, not name-matching. Non-expert linears and embeddings at 8-bit.
• imatrix clip-range refit (activation-weighted) — at 6-bit the min/max range is already optimal (mean shrink 1.000), consistent with this model's measured near-total absence of activation outliers.

Kept unquantized: router + expert bias (fp32), A_log/dt_bias (fp32), KDA conv kernels and f/b/g_proj gates, all norms (fp16) — the state-forming and gating parameters whose error compounds through the recurrence.

The safetensors container is dtype-aligned (every payload offset aligned to its element size), so MLX maps every tensor zero-copy.

Stock mlx_lm has no bailing_hybrid support — this bundle targets the JANG runtime (Osaurus).

Measured quality

Held-out KL vs the merged BF16 source (64 prompts / 21,041 tokens, disjoint from calibration):

Margin-conditioned flip curve: 18.9% → 2.7% → 0.45% → 0.10% → 0.00% → 0.00% across rising source-margin bins — strictly monotone, the lossy-but-healthy signature. (The previous v0.5 build measured 0.0184 / 95.97% on the same prompts against its own source.)

Local agentic bench (20 real multi-step Osaurus tool tasks, verified against real filesystem/sqlite state, XML-arg parser, greedy decode): 14/20 with 0 schema violations — identical to the BF16 source run under the same harness and seed (14/20), with 5 of 6 failures on the same tasks. The quantization introduces no agentic defect; the residual failures are model traits.

Sampling profile (stamped, measured)

generation_config.json and jang_config.chat.sampling_defaults both carry the validated profile:

temperature 0.7 · top_p 0.95 · top_k 20 · repetition_penalty 1.05

This is not a default-copy: at temperature 1.0 this model family intermittently emits prose before tool calls and can enter repetition loops; 0.7 was measured to eliminate the malformed-prose defect. The v2mcp probe measurements above were all taken at this profile. Serve it at this profile.

Reasoning and tool calling

• Reasoning is ON by default (upstream template behavior) with enable_thinking as the off-switch. The template hardcodes preserved_thinking = true, so on its own it retains every prior-turn <think> block. This bundle therefore also stamps reasoning.history_reasoning: "omit" in jang_config.json: it asks the serving app to render each prior assistant turn without its reasoning_content (current-turn content and all tool calls untouched). This suppresses a byte-identical tool-call re-issue loop — once one identical successful turn with its <think> sits in history, the model tends to re-emit it verbatim; dropping historical reasoning breaks that. The cost is that a prior cycle's think tokens (≈11–23 here) are no longer prefix-cache-reusable. The key is a no-op on serving builds that do not yet read it.
• 🚨 Tool calls are XML-arg, not JSON: bare function name then <arg_key>/<arg_value> pairs inside <tool_call>. Schemas go in as JSON; calls come out as XML. No Hermes/Qwen-style parser matches — jang_config.json sets tools.mlx_lm_autodetected: false and names the required bailing_v3_xml_arg parser.

Known limitations

• Asked to delete or remove a configuration entry it cannot verify, the model tends to compose a config write instead of checking first. In Osaurus 0.24.3 every osaurus_config apply in interactive chat goes through the native plan-review approval card (diff + prune warning) and unattended surfaces auto-deny, so such an attempt surfaces as a card to reject or a validator rejection, never as a silent change. Review config cards before approving, especially any that mention prune.
• After a web/tool search the model sometimes re-searches instead of synthesizing. This is a base-model trait; the repetition penalty bounds the visible loop and capping agent iterations in the harness bounds the rest.
• Not positioned as a coding specialist; it is an agentic/tool-use model.

---

Built by Jinho Jang — [email protected]. Base model by inclusionAI (MIT).