LFM2.5-230M

Measured on device (edge-compat, lfm2.5-230m-int4): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 41.6 tok/s · prefill 1101 tok/s · TTFT 210 ms · all 492 ops delegated (2026-09-01); iPhone 17 Pro · LiteRT-LM 0.16.0 · GPU · decode 161.7 tok/s · prefill 3036 tok/s · TTFT 71 ms (2026-09-01); Mac Studio M4 Max · LiteRT-LM 0.16.0 · GPU · decode 555.3 tok/s · prefill 12862 tok/s · TTFT 22 ms (2026-09-01). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/lfm2.5-230m-int4/CARD.md

Measured on device (edge-compat, lfm2.5-230m-int8): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 121.7 tok/s · prefill 3813 tok/s · TTFT 60 ms · all 492 ops delegated (2026-09-01); Pixel 8a · LiteRT-LM 0.16.0 · GPU · decode 39.4 tok/s · prefill 846 tok/s · TTFT 270 ms · all 492 ops delegated (2026-09-05); iPhone 17 Pro · LiteRT-LM 0.16.0 · GPU · decode 143.2 tok/s · prefill 3422 tok/s · TTFT 75 ms (2026-09-01). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/lfm2.5-230m-int8/CARD.md

LFM2.5-230M — LiteRT-LM

LiquidAI/LFM2.5-230M converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. Tested on litert-lm 0.16.x; the bundles carry the ExecutorMetadata section that litert-lm ≥ 0.15 requires for this hybrid architecture.

LFM2.5-230M is Liquid AI's smallest instruction-tuned model (230M parameters, 8 gated short-conv blocks + 6 GQA attention blocks, tied embeddings, 65k vocab). Liquid positions it for data extraction and lightweight on-device agentic pipelines, not for math or heavy reasoning — our measurements below agree.

int8 is the recommended file. It is closer to bf16 on instruction following, and at this model size it is also the faster file on Android GPU (2.8× faster decode than int4 on the Galaxy S26) and equal on desktop. int4 is 89 MB smaller and decodes 13% faster on iPhone Metal — pick it for size or iPhone-only deployments.

Correctness

8-question sanity gate (Apple M4 Max, litert-lm 0.16.0 lineage): int8 7/8 on CPU and 7/8 on GPU; int4 6/8 on CPU and 8/8 on GPU, no degeneration on any leg. The bf16 PyTorch reference scores 8/8 on the same questions. int8's one miss is a rhyme-completion line; int4's CPU leg misses that plus one translation question — a real but small quantization cost.

Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno): both files generate correctly on GPU and CPU, with full OpenCL delegation — every prefill signature and the decode graph replaced by LITERT_CL with zero rejected ops (492/492 nodes on each prefill signature).

iPhone 17 Pro: both files pass the same generation check on Metal and CPU (clean stop, thermal nominal throughout).

Accuracy

The model's headline skill is instruction following, so the quantization was chosen on it. We ran the IFEval prompts through a re-implementation of the mechanically checkable instruction types (n=120, prompt-level strict, greedy). These numbers are comparable only within this table — the checkers are simplified, so do not compare them to published IFEval scores:

int8 is statistically indistinguishable from bf16 on paired prompts. int4 costs ~7 points of instruction following — that is the price of the 89 MB saving. GSM8K is near floor for every configuration, as Liquid's own model card implies; no quantization changes it.

One recipe note: the obvious export-time int8 recipe (which also quantizes the conv layers) measured 53.3% on the same harness — 5 points below shipping int8. These bundles keep the convs float, matching what the LFM2.5-1.2B-JP and 2.6B conversions found for this architecture.

Usage

litert-lm run ./LFM2.5-230M_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."

# GPU
litert-lm run ./LFM2.5-230M_int8.litertlm --backend gpu --cache no --prompt "..."

The bundle embeds the model's own chat template (ChatML role markers, tool-calling branch included) and declares stops <|im_end|> and <|endoftext|>. KV budget is 4096 tokens with 11 prefill signatures (1–1024).

One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's short-convolution state: it reuses the prompt tokens the two conversations share and does not reset the state (my open report). In a probe that asks three questions (What is the capital of France? / What is 17 times 23? / What color is the sky on a clear day?) first in a pirate-persona conversation and then in a conversation told to answer in one word, on the same Engine against a fresh Engine in a new process (answers capped at 200 tokens; CPU backend; litert-lm 0.18.0 on a Mac), the _int8 file answered the capital question with "capital" on the same Engine against "Paris" on a fresh one, in both of its runs (a probe is run a second time only when its first run does not pass). The probe could not measure the _int4 file: its fresh-Engine baseline did not pass the probe's own check (all three expected answers present: Paris, 391, blue) in either of its two runs, so there was nothing to compare. Create a new Engine for each conversation. The first conversation on an Engine is unaffected.

Tool calls on litert-lm 0.18.0. Since the 2026-10-14 re-pack (metadata only: llm_model_type lfm2), the runtime parses the LFM2 tool call and the tool result reaches the model; the earlier files returned the call as message text (<|tool_call_start|>[get_weather(location="Paris")]<|tool_call_end|>). The lfm2 model type needs litert-lm ≥ 0.15.0; the tool-call parser is 0.18.0's — on 0.15–0.17 the call still arrives as text (read from the runtime source, not run here). The 2026-10-14 re-pack changes the model type, and with it the runtime's GPU executor defaults (every non-generic LiteRT-LM model type shares them: allow_src_quantized_fc_conv_ops on, OpenCL hint_waiting_for_completion off, kernel batch size unset); the phone GPU rows on this card were measured with the previous files and have not been re-run — the Mac GPU digit check passed on 0.18.0.

History think blocks: a template property, not observed. This file's chat template keeps a <think>…</think> block only on the assistant turn that follows the latest user message and drops it from earlier turns, while the runtime requires each re-render of the conversation to extend the previous one (my open report). So if a reply ever carried a think block, the second message of that conversation would fail with INTERNAL: The new rendered template string does not start with the previous rendered template string. The model does not open its answers with a think block, and none of the replies in my litert-lm 0.18.0 regression run (Mac, CPU backend, Python API, both files) carried one: the two-turn check answered "Kenji" to "What is my name? Answer with the name only." after "My name is Kenji." on both files. Nothing to do unless you see that error; then keep one message per conversation.

Performance

litert-lm benchmark (litert-lm 0.16.0), Apple M4 Max, -p 256 -d 256 --runs 3 --cache no, quiet machine, ≥300 s rest before each GPU reading:

Galaxy S26 (SM-S942Q, Adreno; litert_lm_advanced_main from the litert-lm v0.16.0 release kit, 205-token prompt with --benchmark, 2 runs per cell, ranges shown):

The int4 GPU prefill range's low end is the cold shader-compile run. Note the ordering: at 230M, int8 decodes 2.8× faster than int4 on this GPU — the blockwise dequant overhead dominates at this size. The larger LFM2.5 models show the reverse.

iPhone 17 Pro (BenchmarkApp yardstick, 128-token prompt, cold runs, one reading per cell):

iPhone Metal is the one backend where int4 out-decodes int8 (162 vs 143 tok/s).

Conversion notes

Converted with litert-torch 0.9.3 / transformers 5.14.1 / ai-edge-quantizer, released wheels only (reproduction scripts: hf-to-litertlm).

• The upstream chat template does not run on the runtime's Jinja engine. LiquidAI's chat_template.jinja uses HF's {% generation %}/{% endgeneration %} training-mask markers (a parse error in minijinja) and message.get("content") (minijinja maps have no .get method). The embedded template strips the two markers and rewrites the two .get calls to plain indexing. Rendering is byte-identical to the original across user/system/multi-turn/tool-calling conversations, verified through HF apply_chat_template.
• KV budget 4096, not 4099. With the 1024-token prefill signature present, a 4099-token KV cache fails GPU engine creation at shader compile (measured on macOS WebGPU); 4096 — a multiple of the widest signature — compiles everywhere we tested.
• Convs stay float in the int8 file (see Accuracy). int8 is applied post-export to linears + embedding.
• ExecutorMetadata added post-export (the released exporter omits it for this state-carrying architecture; litert-lm ≥ 0.15 needs it to bind the 8 conv states and 12 KV buffers). Weights are byte-identical through the edit.
• Stops declared as token ids 7 (<|im_end|>) and 2 (<|endoftext|>); BOS <|startoftext|>. Tokenizer embedded as the upstream tokenizer.json.

License and changes

Distributed under the LFM Open License v1.0 (inherited from the base model; see the license link above). Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and a render-equivalent repair of the chat template repackaged into the .litertlm bundle; runtime metadata (stops, ExecutorMetadata) added. This repository is a community conversion and is not affiliated with Liquid AI.