LFM2.5-1.2B-JP

LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

Measured on device (edge-compat, lfm2.5-1.2b-jp-int4): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · did not run: engine_create_failed (2026-08-24); Raspberry Pi 5 · LiteRT-LM 0.16.1 · CPU, 4 threads · decode 9.2 tok/s · prefill 54 tok/s · TTFT 4.83 s (2026-09-01). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/lfm2.5-1.2b-jp-int4/CARD.md

Measured on device (edge-compat, lfm2.5-1.2b-jp-int8): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · did not run: engine_create_failed (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 41.3 tok/s · prefill 485 tok/s · TTFT 450 ms (2026-09-05); Raspberry Pi 5 · LiteRT-LM 0.16.1 · CPU, 4 threads · decode 7.0 tok/s · prefill 75 tok/s · TTFT 3.56 s (2026-09-01). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/lfm2.5-1.2b-jp-int8/CARD.md

LFM2.5-1.2B-JP — LiteRT-LM

LiquidAI/LFM2.5-1.2B-JP converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime (requires litert-lm ≥ 0.15 / a recent AI Edge Gallery). Sibling of litert-community/LFM2.5-1.2B-Instruct.

Update (2026-08-04): the .litertlm files were updated in place to add the ExecutorMetadata section that litert-lm ≥ 0.15 requires to bind the hybrid conv/attention state buffers (without it, 0.15 fails at inference with missing some output TensorBuffers). Weights and graph are byte-identical to the original release, and the files ran on litert-lm 0.14 until the 2026-10-13 re-pack, whose lfm2 model type needs 0.15.

LFM2.5-1.2B-JP is the Japanese-optimized chat variant of Liquid AI's hybrid conv-attention flagship — a 1.2B model tuned for natural Japanese conversation that runs fully on-device at ~100 tok/s on a laptop CPU.

日本語に最適化された 1.2B オンデバイスモデルです。 スマホの CPU だけで動作し、ネット接続なしで自然な日本語チャットができます。AI Edge Gallery にインポートするだけで試せます。

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

Accuracy

GSM8K (English, greedy, 0-shot CoT, max-tokens 1024, n=100, same harness for all rows): PyTorch bf16 63% · LiteRT int8 65% (full parity, +2pt within noise) · int4 55%. English math undersells a Japanese-optimized tune — it is reported for quantization-fidelity transparency, not as this model's headline; both files pass an 8-question sanity gate (7/8, zero degenerate) and Japanese conversation quality was verified by inspection. Note the int8 file quantizes linears only (convs stay float): quantizing this tune's convs costs ~9pt, unlike the Instruct sibling where conv-int8 is free.

One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's short-convolution state: it reuses the prompt tokens the two conversations share and does not reset the state (my open report). In a probe that asks three questions (What is the capital of France? / What is 17 times 23? / What color is the sky on a clear day?) first in a pirate-persona conversation and then in a conversation told to answer in one word, on the same Engine against a fresh Engine in a new process (answers capped at 200 tokens; CPU backend, also for the _gpu files; litert-lm 0.18.0 on a Mac), the _int4, _int4_gpu, _int8 and _int8_gpu files each gave the same three one-word answers on both Engines, in its one run. Those three short questions say nothing about longer conversations. Create a new Engine for each conversation. The first conversation on an Engine is unaffected.

Tool calls on litert-lm 0.18.0. Since the 2026-10-13 re-pack (metadata only: llm_model_type lfm2, and the chat template now renders tool calls and tool results), the runtime parses the LFM2 tool call and the tool result reaches the model; the earlier files returned the call as message text (<|tool_call_start|>[get_weather(location="Paris")]<|tool_call_end|>). The lfm2 model type needs litert-lm ≥ 0.15.0; the tool-call parser is 0.18.0's — on 0.15–0.17 the call still arrives as text (read from the runtime source, not run here). For _int4_gpu and _int8_gpu, the GPU files: the 2026-10-13 re-pack changes the model type, and with it the runtime's GPU executor defaults (every non-generic LiteRT-LM model type shares them: allow_src_quantized_fc_conv_ops on, OpenCL hint_waiting_for_completion off, kernel batch size unset); the phone GPU rows on this card were measured with the previous files and have not been re-run — the Mac GPU digit check passed on 0.18.0.

Usage

litert-lm run ./LFM2.5-1.2B-JP_int8.litertlm --prompt "海について一文で書いてください。"

GPU — the _int4_gpu file

Pixel 8a (Tensor G3), litert_lm_main built from the litert-lm v0.16.0 release tag, 263-token prompt, generation capped at 256, 3 runs per backend:

On phone-class hardware the GPU's win is prefill and time-to-first-token (3–6× both); decode is bandwidth-bound and roughly a wash, so long prompts gain far more than long answers. The CPU rows spread because the phone throttles across a run series; the GPU rows repeat to within 2%.

Apple M4 Max (litert-lm benchmark --cache no, litert-lm 0.16.0, -p 256 -d 256 --runs 3, otherwise idle machine; both backends were verified to generate correct text before these numbers were quoted):

On a desktop GPU the gain is across the board — roughly 11× prefill and 4× decode against the same file on CPU.

GPU — the _int8_gpu file

_int4_gpu above is the small, fast GPU file. _int8_gpu is the accurate one: the same weights as the int8 file — post-hoc int8 on the linears and embedding, convs left float — re-exported on the same litert-torch 0.9.3 lineage so a GPU delegate will take it. Before it existed there was no way to run this repo's int8 weights on a GPU at all; the int8 file reaches 536 of 579 operations and the runtime refuses the partial split.

Galaxy S26 (SM8850, Adreno), litert-lm v0.16.0, 205-token prompt, generation gated separately from the benchmark:

The two columns were measured a day apart in separate sessions, not side by side. Both are the first of two runs, which is the fair comparison available: this phone slows as it heats, and the two files generate different amounts of text in a benchmark pass, so their second runs are not exposed to the same heat. The spread is shown so you can see it. The decode figures in particular are indicative rather than controlled — the generation lengths differ.

They are a pair, not an upgrade. int4 decodes faster and downloads 41% smaller; int8 answers better. Prefill is close. Pick int4 when tokens per second or download size is the constraint, int8 when accuracy is.

Memory is the surprise. Measured the same way on the same phone — process high-water mark during a generation run — the 1.24 GB int8 file peaked at 556 MB and the 736 MB int4 file at 618 MB. Both figures move a little between runs, and the gap survives that: every reading taken of the int8 file — logged and unlogged — sits below every reading taken of the int4 file. The larger file is the lighter one in use, and neither is anywhere near the several-gigabyte figure a "the GPU holds weights in fp32, so budget about four times the file size" rule of thumb predicts. If you have been sizing GPU headroom that way, measure instead.

Apple M4 Max, litert-lm 0.16.0, -p 256 -d 256 --runs 3 --cache no, each reading taken after a 300 s idle period:

Decode moves with the KV budget here just as it does on CPU — 1024 is the faster setting and a good chat default; the file allows up to 4096. Compare the right rows: the _int4_gpu table earlier in this card was taken at the default 4096 budget, so the like-for-like pair is this file's 4096 row against it, not the 1024 row.

Reproducing these GPU figures. This Mac's GPU slows under back-to-back benchmarking much faster than you would guess. Four runs of one file in a row, at the same effective settings, read 3712 → 3712 → 3529 → 2324 tok/s prefill: flat for two, then a 37% fall, with decode falling further over the same four runs (249 → 247 → 224 → 140). Recovery needs a real pause: 180 s of idle returned prefill to 3712, 300 s to 3946 and 900 s to 3949, so 180 s still leaves you about 6% low. The two rows above were each taken after 300 s, one reading at a time, which is what you need to reproduce them. The _int4_gpu Mac table earlier in this card predates this finding — same --runs 3 protocol, but with no record of what the machine had been doing beforehand — so treat a head-to-head between the two tables as rough rather than exact.

Quality is unchanged by the re-export. Measured head to head against the published int8 file through one harness on one day on the macOS CPU backend, GSM8K n=100 greedy: 65% for both. Asked the same prompt on the CPU backend with greedy decoding, the two files return byte-identical text (144 bytes, character for character) — the re-export changes the graph, not the weights.

GPU — iPhone 17 Pro (iOS Metal)

The engine-creation failure previously tracked in LiteRT-LM#3129 turned out to be a context-sizing issue on the integration side, not a runtime bug. This section is about the _int4_gpu file; the _int8_gpu file has not been tested on iOS. The Metal delegate compiles its kernels against the KV cache that maxNumTokens pre-allocates; a value that does not match the file's exported plan makes engine creation fail with a shader-compile error. On iOS Metal, set maxNumTokens to 1024 (Swift EngineConfig(maxNumTokens:), C API litert_lm_engine_settings_set_max_num_tokens).

iPhone 17 Pro, litert-lm v0.16.0 xcframework, maxNumTokens 1024, single run, device charging: decode 70.0 tok/s, TTFT 91 ms on a short prompt, peak ~560 MB. All Metal kernels compile and generation is coherent.

Run on Android

Install a recent Google AI Edge Gallery, import this repo (or adb push a file and use local import: menu → Models → “+” → From local model file), select the CPU backend, and chat — in Japanese. GPU acceleration needs one of the _gpu files — _int4_gpu or _int8_gpu — and litert-lm ≥ 0.16.0; the Gallery bundles its own runtime, which may lag that version — if its GPU toggle fails there, use CPU.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, CPU backend, -p 256 -d 256 --runs 3 (the tool averages three iterations), warm-up run discarded, otherwise idle machine. Decode on this family depends strongly on the KV budget, so both settings are listed:

Set --max-num-tokens to the smallest value your use case needs — 1024 is a good chat default, and the file allows up to 4096. At 1024 the int4 file decodes fastest; at 4096 the two variants converge.

The _int8_gpu file carries the same weights as the int8 file, and on the CPU backend it measures like it. Run one after the other, -p 256 -d 256 --runs 3 --cache no --max-num-tokens 1024: _int8_gpu 396.9 tok/s prefill / 86.3 tok/s decode against int8 413.8 / 87.1. Decode is within 1%. Prefill is noisier: the same GPU file measured 396.9 here and 409.7 on an earlier pass, a 3.1% spread of its own, which is the same order as the 4.1% gap to the control's single reading — so the two files are not cleanly separated on prefill. No separate CPU table is given for the GPU file — use the numbers in this paragraph, which were taken in one sitting; its GPU figures are in the _int8_gpu section.

A caveat about the table above, and it is about the table rather than about any file: those rows were measured by an earlier session, and the same int8 file re-measured with the command just quoted reads 413.8 tok/s prefill against the 1526 shown, and 87.1 tok/s decode against 98.5. What differs between the two measurements has not been run down, so treat the prefill column above as not currently reproducible and the decode column as roughly right.

Use the CPU backend for the int8 and int4 files — neither can create a GPU engine. Both come from the pre-0.9.2 ShortConv export generation, whose prefill graph still carries INT64 ADD/CAST inside Lfm2ShortConv, plus GATHER_ND and a GREATER_EQUAL with const inputs. The GPU delegate takes 536 of the 579 operations and leaves 43 on the CPU, and the runtime then refuses the partial split: Hint fully delegated to single delegate is set, but the graph is not fully delegated. Re-exporting from the post-0.9.2 lineage removes those INT64 ops — that is what the _gpu files are. _int4_gpu runs fully delegated on Android OpenCL, macOS and iOS Metal (with maxNumTokens 1024; see the iPhone section); _int8_gpu carries these same int8 weights and runs fully delegated on Android OpenCL and macOS.

On a Pixel 8a (Tensor G3, CPU backend, measured in AI Edge Gallery) int8 decodes at ~19 tok/s and int4 at ~31 tok/s — on phone-class memory bandwidth the int4 file is about 1.7× faster as well as 41% smaller, so prefer int4 on mid-range devices. Those Android figures are single ship-gate runs, not medians. First device load compiles the graph and can take about a minute; later loads are instant.

Conversion notes

Converted with released litert-torch 0.9.1 with the same exporter fix as the Instruct sibling: the stock LFM2 short-conv block saves its conv state from padded prefill columns, corrupting the first generated token of nearly every reply; the fix derives the chunk's valid length from the attention mask in-graph and gathers the state from the last valid columns (verified token-identical to an exact per-token reference loop). Multi-length prefill signatures (1–1024). Quantization on this finetune: post-hoc int8 on linears + embedding with the convs left float, and post-hoc int4 on linears only. Post-hoc conv quantization breaks generation. Quantizing convs at export time instead costs accuracy here — 9 GSM8K points on this tune, 56 against 65 — so the int8 and int4 files both leave them float.

Why there is a separate _int4_gpu file. The exporter patch above reads the chunk's valid length with index_select and a mask sum, which lower to GATHER_ND and INT64 ops, and GPU delegates reject both — the CPU-only files take 536 of 579 operations on the GPU and the runtime then refuses the partial split. litert-torch 0.9.2 fixed the underlying conv-state bug upstream using valid-token masking and a one-hot matmul state select, which emits neither op. _int4_gpu is that export: litert-torch 0.9.3 + litert-converter 0.3.1, the same int4 recipe and the same weights as the int4 file, plus the executor-metadata section litert-lm ≥ 0.15 needs. It delegates fully on Android OpenCL — 501/501 and 519/519 nodes, zero rejected ops — and runs on the macOS GPU backend. Quality is unchanged: run head to head against the published int4 file on identical prompts, decoding and scoring, both score 38/50 on GSM8K (CPU, n=50), and the GPU file passes an 8-question sanity gate on both backends. The CPU-only files are left exactly as they are rather than replaced, so nothing changes for anyone already using them.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with litert-lm benchmark 0.16.1: CPU backend, 4 threads, 256 prefill + 256 decode tokens, --cache memory (the compile cache lives and dies with the process, so every invocation compiles the model from scratch; nothing is reused between runs), one warm-up plus one timed iteration per invocation, 3 invocations per file with cooldown in between. Values are the median across invocations (min–max in parentheses). No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0). Every file listed produced coherent text in a real generation on this backend before its numbers were recorded.

Known runtime issues

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders these files' chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on these files' template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.

License and changes

Distributed under the LFM Open License v1.0 (see LICENSE, inherited from the base model). Note the license's commercial-use limitation for organizations above US$10M annual revenue. Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged unmodified; exporter conv-state fix as described in Conversion notes. This repository is a community conversion and is not affiliated with Liquid AI.