LFM2.5-2.6B-MXFP8

<p align="center"><a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a></p>

OsaurusAI/LFM2.5-2.6B-MXFP8

MXFP8 build of LiquidAI/LFM2.5-2.6B — LiquidAI's always-thinking 2.6B agentic model (16 languages, 128K context) in the OCP microscaling FP8 format, with learned codes: activation-aware AWQ folds plus GPTQ codes-only QAT on every FFN tensor. Same size as the vendor's own MXFP8 export, lower KL in both measured domains.

Want the best quality per GB? Take LFM2.5-2.6B-JANG_6M (2.30 GiB, KL 0.033/0.0033 bits vs this build's 0.272/0.113). Choose MXFP8 when you want the OCP microscaling format end-to-end.

Bundle

Measured (M5 Max, stock mlx-lm 0.31, vs bf16 source)

Two 768-token held-out texts: general/encyclopedic and agentic/code+thinking. KL is mean full-vocabulary KL(bf16 ‖ quant).

As on every model we have measured, 6-bit affine beats MXFP8 on fidelity while being smaller — e4m3 elements carry ~3 mantissa bits, so "8-bit MX" is not strictly better than 6-bit affine with a per-group scale and bias. It shows in behaviour too: under pure greedy decoding this build thinks more verbosely than bf16 (it re-verifies its own arithmetic before answering — still correct, </think> closes, eos fires); with the card's default sampling (temperature 0.1) outputs are clean and concise. Runtime gates all pass: greedy math reasoning, card-default sampling coherence, Liquid-format tool calls, grounded 2K-token long-context answers.

Chat / reasoning

• Thinking is ALWAYS on. LFM2.5-2.6B is a pure reasoning model: the chat template unconditionally opens <think> at the start of every assistant turn. There is no enable_thinking switch — the only template kwarg is preserve_thinking (default false: prior turns' reasoning is stripped except after the last user turn).
• The template is shipped verbatim (chat_template.jinja, also inlined into tokenizer_config.json), and capabilities.think_in_template = true is stamped so think-tag parsers route the pre-opened block correctly.
• No BOS trap: the template emits <|startoftext|> itself and the tokenizer never auto-adds one — both apply_chat_template(tokenize=True) and re-encoding the rendered string yield exactly one BOS.
• Stop token eos_token_id = 124900 (<|im_end|>).
• Tool calls use the Liquid Python-call format: <|tool_call_start|>[get_weather(city='Seoul')]<|tool_call_end|> (verified live).
• Sampling defaults (vendor card + generation_config.json, mirrored in jang_config.chat.sampling_defaults and gate-checked against each other at build): temperature 0.1 · top_k 50 · repetition_penalty 1.1.

Usage

Standard MLX safetensors with {"group_size": 32, "bits": 8, "mode": "mxfp8"} in config.json[quantization]. Loads with stock mlx_lm >= 0.31 — no custom code, no trust_remote_code. Runs in Osaurus and vMLX-compatible runtimes (lfm2 family).

from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/LFM2.5-2.6B-MXFP8")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Which number is bigger, 9.11 or 9.8?"}],
    add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024))

---

Quantized and verified by Jinho Jang ([email protected]). Base model © Liquid AI, released under the LFM 1.0 license.