LFM2.5-8B-A1B-CoreAI

LFM2.5-8B-A1B — Core AI (.aimodel)

LiquidAI/LFM2.5-8B-A1B as Core AI .aimodel bundles for Apple silicon, published by visible-cx. They load through Core AI on macOS and are not usable by PyTorch, GGUF or MLX. This is the model the Visible app routes Perspectives and Reports to — the long-prose families, where a 16,384-token window and grammar-constrained decoding matter more than raw speed.

It is a sparse mixture-of-experts model: 8.3B total parameters, ~1.5B active per token. 24 layers — 18 short-convolution mixers and 6 GQA attention layers; the first two layers are dense, every layer after is a 32-expert top-4 MoE. The expert weights are read through a gather_qmm Metal kernel, which is what makes the model fast: upstream measured 140.4 tok/s on an M4 Max through the gather path against 39.2 tok/s for the same weights read as a dense over-read.

aff4/ is the bundle to use. It supersedes mf64/ (the sym8 build) on one arithmetic fact and not on preference: on a 16 GB Mac sym8 requires 12.72 GiB free before it will start, which is more than that machine has ever been measured to have; the aff4 D2 rebuild requires 7.55–7.87 GiB and has completed an 11-question report leg end to end at 56.2 tok/s. mf64/ is kept as the evidence for that comparison, not as a fallback. See Measurements.

Contents

Each folder holds <name>.aimodel/ (main.mlirb, main.hash, asset metadata.json), a bundle-level metadata.json, and tokenizer/ (tokenizer.json, tokenizer_config.json, chat_template.jinja).

aff4/README.md sits beside the bundle directory rather than inside it, deliberately: a downloader that verifies a bundle by summing its subtree — as this project's does — counts a repo card inside the bundle folder as 1,084 unexpected bytes and refuses. Keep repo prose one level above any bundle directory.

Stop token: eos_token = "<|im_end|>" in all three bundles.

Chat template. LFM2.5-8B-A1B is an always-thinking model, so the generation prompt terminates the reasoning block — {{- "<|im_start|>assistant\n<think></think>\n" -}}, the same form the LFM2.5-2.6B bundles use. Free-form generation therefore returns text with the bundle exactly as published rather than spending the whole budget inside an unterminated <think> block. Guided decoding is unaffected either way, because the grammar forces structured output from the first token.

Provenance

BatchedMetalSwitchGLU serves S=64 by sorting (token, expert) pairs and falls back to the q=1 gather path at S=1, so both entrypoints keep the gather kernel and decode arithmetic is identical between them.

aff4/ — the D2 rebuild

D2 is mf64/'s weights re-quantized and re-exported under three changes, each priced separately against an fp32 oracle before it was taken:

The head's quantization scheme is the whole story of this bundle. An affine head (D1) compiled to a 1,457,034,236-byte graph constant; the symmetric head (D2) compiled to 408,458,282. Same weights, same size on disk, one field different. A symmetric dequantize is a scale multiply the delegate absorbs into the matmul; an affine one is (q − zp) × scale, and the subtract is not absorbed — so the folder precomputes two half-gigabyte dequantised 2048x128000 fp16 transposes of the head instead. D1 paid for its head three times. The lever is not "quantize the head", it is "give the head a dequantize the delegate can absorb".

Non-expert weights outside the head are the shipped int8 per-block-32 recipe calibrated on the decode spec. L3 — dropping the prefill entrypoint — was not taken: it is worth 0.27 GiB on this bundle and it would put every prompt token through the S=1 graph.

Not published, deliberately: a non-QAT int4 int4km-gather variant exists upstream and does not hold quality on this model.

Requirements

• Apple silicon Mac, Core AI runtime. Mac-only — neither bundle fits an iPhone even with the memory-limit entitlement.
• Engine contract: 2 inputs (input_ids, position_ids) → logits. No static inputs, no per-step mask. Runs on the pipelined engine and on the logits-capable sequential engine, which is what grammar-constrained decoding needs.
• Manifest context 16,384; maxOutputTokens and the S=64 prefill chunk are read from the bundle's own metadata.json (function_map: {"main": ["main", "prefill"]}).
• The runtime wires the whole weight set at the first forward pass, not at load — an A1B is charged for all 8.3B parameters resident. "1.5B active" is a decode-speed property and nothing else. Budget by the numbers in Measurements, not by active parameter count.
• Minimum practical machine memory: 16 GB for aff4/, with the machine reasonably quiet. mf64/ does not start on 16 GB; 24 GB or more.
• Disk: the bundle plus its compiled graph cache on first load. aff4/ compiles to 6,115,467,248 B; mf64/ to 10.374 GiB.
• The bundle manifest declares runtime_env COREAI_CHUNK_THRESHOLD = "1", the most memory-conservative prefill setting available. Keep it.

Measurements

All figures below are from a 16 GB Apple silicon Mac (M2 Pro, macOS 27 beta), under an external watchdog sampling wired memory, reclaimable memory and the compressor. Compile-only figures are zero-prompt runs with no forward pass.

aff4/ (D2) — the completed leg

Eleven real report questions, guided JSON-constrained decoding against a fixed schema, greedy, both kill criteria armed for the whole run and verified alive.

Two later runs on the same bundle read 55.81 and 59.81 tok/s, so 56.23 sits in the middle of a narrow band rather than at a lucky edge.

Throughput is flat in prompt length — question 11 ran at 55.12 tok/s against question 1's 59.17, and the difference tracks prompt length rather than elapsed time. Across 521 samples in an earlier leg the longest prompt produced the smallest boundary spike. That flatness is an LFM-MoE property; the dense Gemma 4 12B measured on the same machine does not have it.

Quality on that leg, counted by an automated scorer with every flag hand-adjudicated:

Zero inversions across eleven questions is the headline. One of the eleven answers is bad in a way that matters, and it is recorded rather than averaged away: on a "red flags" question the model recited the question's own enumerated category list back as findings, manufacturing a serious accusation out of the prompt's taxonomy. Ten of eleven are sound; that one is a different order of defect from an untidy one.

Release and re-acquire. Three teardown cycles: wired at peak stable to 0.012 GiB across cycles; wired after release settles to within 0.075 GiB of baseline; warm re-acquire 0.004–0.006 s; output byte-identical across cycles, so no state leaks across a release.

Loading does not wire. After a 116.28 s cold-cache load the wired counter read −0.031 GiB against baseline — the weights are read, and the wiring burst happens at the first forward pass.

The requirement, and why mf64/ is superseded

Compile-only, one at a time, watchdog armed:

This machine's ordinary reclaimable memory is 4.5–7.3 GiB, with a single best-ever reading of 9.97 GiB that has never repeated. 12.72 GiB is above anything it has ever had free. sym8 is not a slow model here; it is a model that cannot start — and the whole 4.2× gap between the two graph constants is compiler-materialised copies that were never weights.

Two honest notes against ourselves:

• The shipped gate is 7.55, the completed run says 7.87. The blob law is 1.3% conservative on wired growth; the 0.32 GiB gap is the dirty footprint, which a ratio over a compiled blob has no term for. A finished measurement raised this model's requirement.
• D2's exact stack was never itself an oracle row. Its components were, separately, and both are free: against an fp32 oracle (41 headline positions, 328 aggregate) the aff4 base scores 4/41 and 79/328; + head int8 symmetric scores 4/41, 79/328; + fp16 attention scores 4/41, 78/328. For comparison sym8 scores 1/41, 14/328 and fp16 scores 0/41, 12/328 — so sym8 is the more faithful quantization, and it is not the one that runs. That trade is the reason this repo keeps both.

mf64/ (sym8) — kept as evidence

Measured before it was superseded, 10-sample guided harness, sequential engine:

Read Max RSS carefully: a resident set counts clean mapped pages the kernel can drop for free, so it was never a statement about what the machine has to give up. The wired figures above are.

mf64-tp/ — RETIRED

Failed its Mac gate on 2026-08-18. Do not deploy it and do not route traffic to it.

A token-tiled prefill kernel. Three headline claims, two failed. (1) The Metal kernel compiles and runs — passed. (2) It is not bit-identical to mf64: 3 of 10 guided rows diverge, including one that invents an entity and one that flips a schema enum, while a same-session mf64 re-run reproduced its own outputs byte for byte on all 10 rows. Both bundles parse 10/10 and are enum-clean, so a schema-shape gate would not have caught this. (3) Prefill came in at 5.35 ms/prompt-token against a prediction of 0.97 — a 1.17× speedup, not 6.5×. Decode is genuinely unchanged, confirming the q=1 fallthrough is clean.

Usage

Swift Package Manager, via CoreAIKit — a community package, not affiliated with Apple, requiring macOS 27 beta:

.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKit", package: "coreai-kit")

ModelID addresses a bundle as repo + path + revision, where path is the subtree in this repo holding one complete bundle (metadata.json + *.aimodel/ + tokenizer/). It downloads from the Hub on first use and is cached afterwards:

import CoreAIKit

let model = ModelID(
    "visible-cx/LFM2.5-8B-A1B-CoreAI",
    path: "aff4/lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2",
    revision: "c9b541e85b9125ecb9dc7f4fa6d3e3c06ccd78e5")   // pin, don't ride main

var config = ChatSession.Configuration()
config.engineVariant = .sequential   // required for guided / grammar-constrained decoding
config.temperature = nil             // greedy

let chat = try await ChatSession(model: model, configuration: config)
for try await event in chat.streamResponse(to: "…") {
    if case .response(let delta) = event { print(delta, terminator: "") }
}

ChatSession(bundleAt:) loads a bundle directory already on disk, which is the path to use if you stage the bundle yourself. Leave COREAI_CHUNK_THRESHOLD alone — the manifest sets it.

Digests

aff4/lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2, computed from the exact published bytes. main.hash is the raw 32 bytes of sha256(main.mlirb) — verified, not assumed.

Why per-file digests and not just a revision. A revision pin is an assertion about a repository; this is an assertion about bytes, and only the second survives a mirror, a re-upload or a CDN edge that lies. Hugging Face stores LFS objects under their SHA-256 and everything else under its git blob oid, so the whole subtree above can be checked against this table with zero bytes downloaded, from the tree API alone.

This bundle has had three addresses in one day — staged locally, published at 9e023baf…, republished at c9b541e8… when a stray README moved out of the bundle subtree — and ad57774b86dae371… did not move once. Uploading bytes does not rehash them.

Core AI .aimodel bundles are not byte-reproducible: the exporter is not deterministic even against itself. Integrity rests on these digests, not on a rebuild.

Status

No PSNR or numerics gate has been run against a PyTorch reference on device for any bundle here. The oracle rows quoted above are box-side torch fp32 comparisons; qualification on the Mac is behavioural (parse rate, enum conformance, grounding scoring, clean stop, needle recall) plus the memory instrumentation.

License

LiquidAI/LFM2.5-8B-A1B is released under the LFM Open License v1.0 (lfm1.0), and upstream declares it as license: other + license_name: lfm1.0. These bundles are a derivative of that checkpoint and the same licence and obligations travel with them — see the upstream licence. Redistribute the licence with the files. The contribution here is the conversion, the compression recipe and the qualification evidence, not the weights.

pf-lastrow/ — the prefill entrypoint the engine can actually dispatch

The aff4-c128000/ bundle already registered a prefill entrypoint. Its shape was the problem — read off the asset itself:

aff4-c128000/   main 1x1 -> logits 1 x  1 x 128000     prefill 1x64 -> 1 x 64 x 128000
pf-lastrow/     main 1x1 -> logits 1 x  1 x 128000     prefill 1x64 -> 1 x  1 x 128000

The engine's dispatch consumes last-row-only logits, so a [1,64,V] prefill is not the function it looks for and a prompt goes through main one token at a time. This revision is the same recipe emitting the row the engine reads.

Per 64-token chunk the logits buffer goes 64 x 128000 x 2 B = 16.4 MB to 256 KB, and the head matmul goes 64 rows to 1 — rows 0..62 were never read.

The recipe is D2, unchanged, and that is the gate. Affine-int4 experts at K-block 64, untied head at int8 symmetric, fp16 attention, --max-ctx 128000. Only the narrow moved:

main.mlirb   aff4-c128000/   5,707,008,988
             pf-lastrow/     5,707,009,429      +441 B

+441 bytes, against this exporter's ~22,000-byte run-to-run noise floor. The decode path is byte-identical in the sense that floor defines.

finish_bundle PASS: producer coreai-core 1.0.0b2, ctx=128000, function_map {'main': ['main','prefill']}.

Not yet run on a Mac. No token has been generated from this bundle and no speed is claimed.

pf-lastrow/lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2_c128k_pf — 5,724,920,876 bytes

lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2_c128k_pf.aimodel/main.hash            32 B  6666eb6ac79b92ae2c3c9c5bf57235c0b95a4afe5b4e8806301f101abe1c977d
lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2_c128k_pf.aimodel/main.mlirb 5,707,009,394 B  6abd852f23dc874e25e6daec53aeaa1edcbbc15fe2414e8fef5e8000d417b272
lfm2_5_8b_a1b_decode_aff4_gather_mf64_d2_c128k_pf.aimodel/metadata.json           105 B  f274d903ee4116e9deac8ff23aae38789740828d56c828f8a48a50b9a19afa80
metadata.json                                                  653 B  6faa3c0aee916bff298c4539b5332d95bcbc5fa62a013bceb673e6ac17dfa238
tokenizer/chat_template.jinja                                4,779 B  b2881deb506bdff56bf6ac46aaa33292c119953f82fedaf276634fcc4282a559
tokenizer/tokenizer.json                                17,905,598 B  695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41
tokenizer/tokenizer_config.json                                315 B  b53c3caceb7fa0de424e1aa67b3cc5008cd2f8b8d2426a8fa221ef3cf1a949d5