LFM2.5-2.6B-JANG_6M

<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

OsaurusAI/LFM2.5-2.6B-JANG_6M

JANG_6M build of LiquidAI/LFM2.5-2.6B — LiquidAI's always-thinking 2.6B agentic model (16 languages, 128K context), quantized for Apple Silicon with 6-bit affine FFN weights, 8-bit token-mixing operators, and learned codes: activation-aware AWQ folds plus GPTQ codes-only QAT on every FFN tensor. This is the quality build of the pair — smaller and more faithful than MXFP8.

Bundle

Measured (M5 Max, stock mlx-lm 0.31, vs bf16 source)

Two 768-token held-out texts: general/encyclopedic and agentic/code+thinking (the workload LiquidAI built this model for). KL is mean full-vocabulary KL(bf16 ‖ quant).

Decode 156 tok/s greedy (bf16 source: 83 tok/s in the same harness). Runtime gates all pass: greedy math reasoning with clean <think>…</think> close and eos stop, card-default sampling coherence, Liquid-format tool calls, grounded 2K-token long-context answers.

Profile comparison

On general text the 6-bit builds are within sample noise of each other; on the agentic/thinking distribution this build's KL is 2.5× lower than the vendor's 6-bit RTN export — that is what the QAT/AWQ calibration is aimed at. And as on every model we have measured, 6-bit affine beats MXFP8 on fidelity while being smaller: e4m3 elements carry ~3 mantissa bits, so "8-bit MX" is not strictly better than 6-bit affine with a per-group scale and bias.

Chat / reasoning

• Thinking is ALWAYS on. LFM2.5-2.6B is a pure reasoning model: the chat template unconditionally opens <think> at the start of every assistant turn. There is no enable_thinking switch — the only template kwarg is preserve_thinking (default false: prior turns' reasoning is stripped except after the last user turn).
• The template is shipped verbatim (chat_template.jinja, also inlined into tokenizer_config.json), and capabilities.think_in_template = true is stamped so think-tag parsers route the pre-opened block correctly.
• No BOS trap: the template emits <|startoftext|> itself and the tokenizer never auto-adds one — both apply_chat_template(tokenize=True) and re-encoding the rendered string yield exactly one BOS.
• Stop token eos_token_id = 124900 (<|im_end|>).
• Tool calls use the Liquid Python-call format: <|tool_call_start|>[get_weather(city='Seoul')]<|tool_call_end|> (verified live).
• Sampling defaults (vendor card + generation_config.json, mirrored in jang_config.chat.sampling_defaults and gate-checked against each other at build): temperature 0.1 · top_k 50 · repetition_penalty 1.1.

Usage

Standard MLX safetensors with a per-module {bits, group_size, mode} map in config.json[quantization]. Loads with stock mlx_lm >= 0.31 — no custom code, no trust_remote_code. Runs in Osaurus and vMLX-compatible runtimes (lfm2 family). Fits in 8 GB of unified memory.

from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/LFM2.5-2.6B-JANG_6M")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Which number is bigger, 9.11 or 9.8?"}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024))

---

Quantized and verified by Jinho Jang ([email protected]). Base model © Liquid AI, released under the LFM 1.0 license.