Agents-A1-4B

Agents-A1-4B — LiteRT-LM

InternScience/Agents-A1-4B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. These bundles contain the text decoder only (vision tower dropped). Both use the same float export, preserve the model's thinking and native XML tool-call format, and declare fp32 activations. Requires litert-lm ≥ 0.15 (the state-buffer binding of the hybrid architecture); every gate below was run on litert-lm 0.17.1 and on the 0.18.0.dev20260919 nightly, and the 2026-10-16 files were checked on litert-lm 0.18.0 and 0.15.0 on a Mac.

Agents-A1-4B is InternScience's agentic model built on the Qwen3.5-4B hybrid architecture: 24 GatedDeltaNet (gated delta rule linear attention) layers interleaved with 8 gated full-attention layers, so memory stays nearly flat with context length — only the 8 attention layers keep a KV cache (4096-token budget here).

2026-10-13: the int8 file re-quantized. The 48 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) now remain float, and the rest of the recipe is unchanged; the chat template, tokenizer and executor metadata are byte-identical. Agents-A1-4B_int8.litertlm: previous file sha256 0001573f…, 4,407,426,400 B → the 2026-10-13 file sha256 1481f76d…, 4,419,086,560 B (+11,660,160 B). Its float export, quantized with the previous recipe, reproduces the previous file's TFLite section byte for byte. The change fixes two things on the CPU backend, measured on a Mac M4 Max CPU (litert-lm 0.18.0, greedy (top-k 1), one fresh process per run, 400-token cap). With thinking disabled, the previous file gave the right product on the six multiplication questions below in 18 of 18 runs but ended by itself in 4 of 18, running on into self-talk until the cap in the other 14; this file ends by itself with the right product in 18 of 18. On the arithmetic prompt (17 × 23) with thinking disabled, the previous file ended by itself in 1 of 3 runs and this file in 3 of 3. With thinking on and no budget (the bundle's default), the previous file's thought did not close in 3 of 3 runs of the arithmetic prompt; this file closes it and answers in 2 of 3. The third run is a long reasoning cut by the cap, with no repeated line; a run with a 1000-token cap closes the thought and answers. I am sorry: since 2026-09-21, anyone who ran the int8 file on a CPU could get a reply that ran on after the right answer until the token cap, or, with thinking on, a thought that did not close. This card's Galaxy S26 row recorded the run-on and sent Android users to the mixed-int4 file. The card's line that the same file "passes every gate above on the Mac" missed the same run-on on the Mac CPU, because my Mac checks of that run-on used the bundle's default sampler (top-k 20, top-p 0.95, temperature 0.85); it shows under greedy (top-k 1) decoding. This was my recipe's defect and my check's miss. The Mac GPU did not show it: with the previous file, the six questions (greedy (top-k 1), thinking disabled, one run each) ended by themselves with the right product in 6 of 6 runs, in 18–19 characters, and the 8-question gate (temperature 0) passed 8/8 there. From now on, when one file of a recipe is found defective, I re-check its sibling files the same day with the six multiplication questions in three fresh processes each under greedy (top-k 1) decoding; a run that reaches the token cap counts as a failure even when its product is right; and I check this card's lines mechanically before publishing it.

2026-10-16: the tokenizer section of both files replaced with the repository's own tokenizer.json (InternScience/Agents-A1-4B at 945c40a4, 19,989,343 B, sha256 87a7830d…); the TFLite section, the executor metadata and the LLM metadata (chat template included) are byte-identical. The previous section was the tokenizer as transformers writes it for the Qwen2Tokenizer class (19,989,325 B, sha256 06b95093…): the same vocabulary, merges and 33 added tokens, but a pre-tokenizer without the repository's \p{M} class, so text with combining marks was split differently from the repository's tokenizer: 49 tokens instead of 36 on a Hindi test sentence, 24 instead of 12 on Thai, 29 instead of 16 on Tamil, 30 instead of 29 on Arabic with harakat. English, numbers, code, chat and tool-call markup, Chinese, Japanese, Korean and emoji tokenize the same, and so does every prompt of the checks and benchmarks on this card. Agents-A1-4B_int8.litertlm: previous file sha256 1481f76d… → this file sha256 53fb9740…; Agents-A1-4B_mixed_int4.litertlm: previous file sha256 f12a9764… → this file sha256 13c53e3a…; both sizes are unchanged (4,419,086,560 B and 2,754,365,536 B: the new section is 9 bytes shorter and the sections keep their offsets), and the mixed-int4 file's container header moves from format 1.6.0 to 1.7.0, the version the litert-lm 0.18.0 packer writes. The checks run on these files are the 2026-10-16 item in the Correctness list; every other result on this card was measured on earlier files, as each item says, and where an item dated before 2026-10-16 says "this file" or "the previous file", it means the file of its date or the one before that; the int8 file of 2026-10-13 and the mixed-int4 file of 2026-09-21 have the TFLite section of the current files byte for byte. I am sorry: every version of these files since 2026-09-21 carried that tokenizer, so text in Hindi, Thai, Tamil, Arabic with harakat or another script with combining marks reached the model as other token ids than the repository's tokenizer gives (what that did to replies was not measured), and my checks, whose prompts tokenize the same with both, did not show it.

Correctness

Measured on Apple M4 Max (the multi-step arithmetic item also on a Galaxy S26). The items down to Native XML tools used the mixed-int4 file of 2026-09-21 (sha256 f12a9764…), the int8 file of 2026-09-21 (sha256 0001573f…) or their common float parent; the multi-step arithmetic item used the int8 file of 2026-10-13 (sha256 1481f76d…), and the 2026-10-16 item the current files:

• Teacher-forced logits parity: fp32 LiteRT CPU versus fp32 Hugging Face, 48 positions; top-1 and top-5 agreement 48/48, minimum per-position Pearson 0.99999999986, mean KL ≈ 0 (−1.6e−8 nats, roundoff), maximum absolute logit difference 2.2e−4; all logits finite.
• Question gate: 8/8 correct, no degenerate or off-topic answers in all eight cells: both files, CPU and GPU, under litert-lm 0.17.1 and 0.18.0.dev20260919.
• Prompt-length sweep: each file passes 40/40 CPU lengths and 20/20 GPU lengths under 0.17.1, with one fresh process per length.
• GSM8K (first 100 test questions): int8 93/100, mixed-int4 93/100 — 0-shot step-by-step prompt, greedy, thinking disabled, GPU on 0.17.1, one fresh process per question, 4096-token context with no separate decode cap (context-limit hits: int8 1, mixed-int4 2). Same prompt and answer extraction as our Qwen3.5-4B card; that run used a 2048-token budget.
• Multi-turn: the int8 CPU three-turn probe answers OK, recalls Osaka, then recalls Ken, with thinking on and off on both runtimes; no prefix-contract exception.
• Thinking: the int8 CPU arithmetic and factual probes answer correctly on both runtimes, report reasoning in the thought channel, and keep literal think tags out of the answer. A thinking budget of 64 shortens reasoning while retaining the correct answer.
• Native XML tools: 12/12 prescribed requests emit one correctly named, well-formed call with plausible arguments and no suffix, using the bundle's template. This validates emitted XML; it does not imply runtime tool-event parsing.
• Multi-step arithmetic, CPU and GPU, the int8 file of 2026-10-13 (one fresh process per run; greedy (top-k 1) on the Mac):
  • Mac CPU (litert-lm 0.18.0, thinking disabled, 400-token cap): six multiplication questions (17 × 23, 23 × 17, 12 × 34, 9 × 87, 45 × 12, 36 × 25), three runs each. 18 of 18 runs end by themselves with the right product, in 18–19 characters.
  • The arithmetic prompt, "What is 17 times 23?", with thinking disabled: 3 of 3 runs answer "17 times 23 is 391." and end there.
  • Thinking on, no budget (the bundle's default), 400-token cap: 2 of 3 runs close the thought and answer 391. The third is a long reasoning cut by the cap, with no repeated line; one run with a 1000-token cap closes the thought and answers 391.
  • Thinking on, no budget, through a raw <think> prefill (400-token cap): 3 of 3 runs close the thought with </think> and answer 391.
  • A 300-token thinking budget (400-token cap): 3 of 3 runs return the reasoning on the thought channel and answer "17 times 23 is 391.", ending there.
  • The 8-question gate (the litert-lm run 0.18.0 CLI, temperature 0): 8/8 on Mac CPU and 8/8 on Mac GPU.
  • Mac GPU (litert-lm 0.18.0, thinking disabled, 400-token cap): the six questions, one run each. 6 of 6 end by themselves with the right product, in 18–19 characters.
  • Galaxy S26 CPU (the LiteRT-LM Android CLI build of 2026-09-18, which has no switch for sampling or thinking): the arithmetic prompt at the bundle's default, thinking on and the bundle's default sampler (top-k 20, top-p 0.95, temperature 0.85), 1024-token cap, in 4 processes. In 4 of 4 the thought closes and the reply is "17 times 23 is 391."
• Tokenizer section (2026-10-16, both current files, Mac M4 Max). The runtime's Engine.tokenize gives the same ids as the tokenizers library with the repository's tokenizer.json on 174 of 174 test strings (21 sentences, Hindi, Thai, Tamil and Arabic with harakat included, and every prompt of the checks on this card) on litert-lm 0.18.0 (GPU) and 0.15.0 (CPU). With thinking disabled, the 8-question gate answers 8/8 on CPU and on GPU (litert-lm 0.18.0, one fresh process per question), a three-turn conversation completes on the CPU, every GPU subgraph is delegated with no validation error, and litert-lm 0.15.0 answers a plain prompt on the CPU.

One int8 answer written as 20.00 is scored wrong by the extractor's trailing-zero handling; with numeric normalization of the same saved answers both files score 94/100.

Usage

The first command below runs with thinking on, the default; the reasoning comes back in the thought channel. --thinking false turns thinking off. --thinking true --thinking-budget N keeps thinking on and limits the reasoning to a budget of N thinking tokens. --preset ./tools.py adds your application's CLI preset, the file that defines its available functions.

litert-lm run ./Agents-A1-4B_int8.litertlm --backend gpu --cache no \
  --prompt "What is the capital of Japan?" < /dev/null

litert-lm run ./Agents-A1-4B_int8.litertlm --backend gpu --cache no \
  --thinking false --prompt "What is the capital of Japan?" < /dev/null

# Replace N with your thinking-token budget.
litert-lm run ./Agents-A1-4B_int8.litertlm --backend gpu --cache no \
  --thinking true --thinking-budget N --prompt "Explain your answer." < /dev/null

litert-lm run ./Agents-A1-4B_int8.litertlm --backend cpu --cache no \
  --preset ./tools.py --thinking false \
  --prompt "What is the weather in Tokyo right now?" < /dev/null

Use the mixed-int4 filename in the same commands to select that bundle. The application must extract and execute native XML calls itself; the CLI preset supplies tool definitions to the prompt.

Chat template

The embedded template accepts message content both as a string and as a sequence of typed parts. The vendor's default system prompt is embedded verbatim, including Current date: 2026-07-14; any user-supplied system message replaces it. When tools are supplied, their definitions precede the system text.

Tool calls use the model's native XML form, <tool_call><function=...> with <parameter=...> elements. The runtime renders the tool definitions into the prompt but does not parse this native XML into tool_calls events. An application parses the emitted block and returns tool results. Assistant history renders position-independently, without inserting a think block into historical turns, preserving the incremental conversation prefix contract.

Performance

Apple M4 Max, litert-lm 0.17.1, litert-lm benchmark -p 256 -d 256 --runs 3 --cache no. Each row is the CLI's result over three measured runs after its default warmup, on a quiet machine (1-minute load under 3.0), with at least 300 seconds of rest before every GPU cell. TTFT is the CLI's inference measurement; engine initialization is reported separately by the benchmark and is not included in this table.

CPU and GPU generation were verified independently by the question gates before benchmarking. These rows and the Galaxy S26 rows below were measured before the 2026-10-16 tokenizer replacement: the M4 Max int8 rows on the int8 file of 2026-09-21 (sha256 0001573f…, the graph before the 2026-10-13 re-quantization), the mixed-int4 rows on the mixed-int4 file of 2026-09-21 (sha256 f12a9764…), whose TFLite section the current mixed-int4 file carries byte for byte.

The phone's GPU path is verified for generation (below) but its speed is not measured yet. iOS is not measured for these files.

Galaxy S26 (Android)

Galaxy S26 (SM-S942Q, Snapdragon SM8850, Android 16) with the LiteRT-LM Android CLI (litert_lm_advanced_main, build of 2026-09-18). Gate = the same 8 questions with thinking disabled: one fresh session per question, the prompt rendered from the bundled template, the bundle's default sampler, 256-token cap. A separate probe keeps thinking on (2048-token cap) and checks that the final answer arrives after the thought.

The 2026-09-21 int8 file, run right after on the same phone in 3 fresh processes under the same conditions (thinking on, the bundle's default sampler, 1024-token cap), gave an answer outside the thought in 0 of 3: each process's whole output was the thought, ending with <answer>17 * 23 = 391</answer>, and the answer text was empty.

On the Galaxy S26 both files run on the CPU. The 2026-10-13 int8 file peaked at 5,477 MB on the phone's CPU and ran at 157.5 tok/s prefill and 9.33 tok/s decode, with a TTFT of 1.77 s and a cold engine init of 22.1 s (benchmark of 256 prefill and 64 decode tokens, median of iterations 2–3). The phone capped its CPU clocks in every iteration, so these are its capped speeds. The int8 file's GPU engine must not be started on this phone: the 2026-09-21 file's start rebooted it, and neither the 2026-10-13 nor the current int8 file was started there.

• On Android I have run these files on the Galaxy S26 (Adreno) only, not on a Mali or 8 GB-class phone. The Qwen3.5-4B Mixed INT4 file is the sibling of this mixed-int4 file: the same architecture, recipes, prefill signatures and size (2.754 GB). On 2026-10-12 its GPU engine build on a Pixel 8a (Arm Mali, 8 GB) drained the phone's free memory until Android began closing other apps, and I stopped the build before an engine existed. Full delegation on the S26 (table above) does not show that the engine builds on another phone: the Pixel 8a run fully delegated the subgraphs it reached and still had no engine when it was stopped. On the Galaxy S26 CPU the 2026-09-21 int8 file peaked at 6.40 GB and the 2026-10-13 one at 5,477 MB (table above). The Qwen3.5-4B int8 file has this file's recipe and size (4.419 GB), and its card reports one process of it on a Pixel 8a CPU on 2026-10-13: the process answered the arithmetic prompt and peaked at 5,407,976 KiB, and the phone's free memory stayed at or above 3,542,768 KiB. I expect this file to behave the same there, but I have not run it on that phone. On an 8 GB-class phone, use the CPU backend on 8 GB phones; the Qwen3.5-4B Mixed INT4 GPU build got no engine on the Pixel 8a. On an 8 GB-class phone the mixed-int4 file on the CPU is the one to try, since the sibling's GPU build got no engine on the Pixel 8a; its Galaxy S26 CPU peak of 4.94 GB (table above) is above that free-memory figure too, so it may not fit either, and this is not measured.
• One Engine per conversation. Turns inside one Conversation keep their context, as the multi-turn probe above shows. A second Conversation opened on the same Engine continues from the first one's recurrent state, so its replies can follow the first conversation's system prompt instead of its own: the runtime reuses the prompt tokens the two share and does not reset that state (my open report). I measured this on a Mac CPU with the Qwen3.5-4B Mixed INT4 and Qwen3.5-2B int8 files, not with these files. A process that runs one conversation is unaffected.
• LiteRT-LM's CPU backend quantizes the input of every int8 linear layer to int8 at run time (XNNPACK dynamic quantization). In the 2026-09-21 int8 file (sha256 0001573f…) the linear projections feeding the delta rule (in_proj_a / in_proj_b) were int8, and its replies on the CPU could run on after the right answer. On a Mac M4 Max CPU (litert-lm 0.18.0, greedy (top-k 1), one fresh process per run, 400-token cap) with thinking disabled, it gave the right product on the six multiplication questions in 18 of 18 runs and ended by itself in 4 of 18. The other 14 ran on into self-talk until the cap, repeating a line such as "I can help you with that. 23 × 17 = 391." or re-emitting the system prompt. On the arithmetic prompt with thinking disabled, 2 of 3 runs ran on the same way. With thinking on and no budget (the bundle's default), the thought did not close in 3 of 3 runs: each repeated a numbered **Final Answer:** 17 times 23 is 391. line until the cap, with no answer text. Through a raw <think> prefill, 1 of 3 runs ended by itself with an <answer>391</answer> tag and no </think>, and the other 2 did not close. With a 300-token thinking budget, 3 of 3 runs began the answer with 391; 2 ended there, and 1 ran on to the cap with a numbered **Final Answer:** line. On the Mac GPU the six questions (thinking disabled, one run each) ended by themselves with the right product in 6 of 6 runs. The 8-question gate (temperature 0) passed 8/8 on the Mac CPU and GPU with this run-on present. My 2026-09-21 Mac checks of the phone's run-on used the bundle's default sampler (top-k 20, top-p 0.95, temperature 0.85) and found no loop in 120 replies on CPU and GPU, so this card did not report it for the Mac. This is the same cause and the same fix as in the 2026-10-11 note on the Qwen3.5-2B card: this file keeps those 48 projections, in the 24 linear-attention layers, float and changes nothing else (+11,660,160 B). Its float export, quantized with the previous recipe, reproduces the previous file's TFLite section byte for byte. Measured on the 2026-10-13 int8 file (Correctness list above; one fresh process per run): on the Mac CPU under greedy (top-k 1) decoding, 18 of 18 runs of the six questions and 3 of 3 of the arithmetic prompt end by themselves with thinking disabled; with thinking on and no budget, the thought closes in 2 of 3 runs at a 400-token cap, in 1 of 1 at a 1000-token cap and in 3 of 3 through a raw <think> prefill; a 300-token thinking budget answers in 3 of 3; the 8-question gate passes 8/8 on CPU and GPU; the Mac GPU gives 6 of 6; and on the Galaxy S26 CPU at the bundle's default (thinking on, the bundle's default sampler, 1024-token cap), 4 of 4 processes closed the thought and answered. The 2026-09-21 file's Galaxy S26 rows stay in the table above as its record.

Conversion notes

The conversion rail uses litert-torch 115a136 plus the Qwen3.5 hybrid patch, float export, post-hoc quantization, and metadata repacking with state-buffer bindings — the same rail as litert-community/Qwen3.5-4B. Both bundles declare fp32 activations, export prefill signatures 1024/256/64/16/4/1, and have a 4096-token cache. Stop IDs include both the end-of-text and end-of-turn tokens. Scripts and a one-command reproduction: hf-to-litertlm (agents_a1_work/, REPRODUCE.md). Since 2026-10-13 the int8 recipe keeps the 48 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) float. The int8 file's exact bytes need the 2026-09-21 environment, the litert-torch 0.9.3 package (with ai-edge-quantizer 0.8.0, ai-edge-litert 2.1.6, torch 2.12.1 and transformers 5.14.1) beside the pinned litert-torch 115a136 checkout and the checkpoint read from a local directory; beside litert-torch 0.9.2, with the checkpoint read by its Hub id, the same recipe gives a TFLite section 2,064 bytes larger, and I did not isolate which of the two differences moves the bytes.

Base-model license: Apache-2.0. Conversion distribution: Apache-2.0.

Known runtime issues

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders these files' chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on these files' template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.