LFM2.5-2.6B-CoreAI

LFM2.5-2.6B — Core AI (int8)

An int8 Core AI conversion of LiquidAI/LFM2.5-2.6B for Apple silicon. This repository contains no trained weights of its own: it is a quantized format conversion of Liquid AI's model, and all model credit belongs to Liquid AI.

Runtime status: runnable with public tooling, but not with an unmodified apple/coreai-models checkout. LFM's fixed-shape convolution state requires two patches from the public, community-maintained john-rocky/coreai-model-zoo. Exact public commits and commands are below.

Converted from source revision dca1825886789bd40b94368f53b1d9ada4c94598. Both upstream safetensors shards were SHA-256 verified against that revision before conversion.

Runtime requirements

This graph has the normal growing key/value cache plus one fixed-shape conv_state. The stock pipelined engine handles only the KV pair. Apply these two public community patches—no per-token-input, static-input, or prefix-cache patch is required for this model:

1. apps/coreai-shared-product.patch
2. apps/coreai-pipelined-extra-states.patch

Both come from john-rocky/coreai-model-zoo@95a29d41, a public community repository maintained by Daisuke Majima (john-rocky). It is not an Apple repository. The patches target Apple's public coreai-models runtime at 5ed9981303b38d5a44aa6b45509bc4f6945029f5.

git clone https://github.com/apple/coreai-models.git
git -C coreai-models checkout 5ed9981303b38d5a44aa6b45509bc4f6945029f5

git clone https://github.com/john-rocky/coreai-model-zoo.git
git -C coreai-model-zoo checkout 95a29d41affed3bdf3ea5992ece094a908e21a04

git -C coreai-models apply ../coreai-model-zoo/apps/coreai-shared-product.patch
git -C coreai-models apply ../coreai-model-zoo/apps/coreai-pipelined-extra-states.patch
swift build --package-path coreai-models -c release --product llm-runner

The public community zoo also supplies the LFM2 exporter and its coreai-models overlay. The small toolkit published here contains the 2.6B-specific wrapper, tokenizer/config corrections, quality gates, and locked Python environment used for this artifact. Full reproduction details are in RECIPE.md.

Which file do I want?

The repository root is the portable bundle. The runtime wants a directory containing metadata.json, the *.aimodel/ graph and tokenizer/, and those sit at the top level here. So the recommended download is the root minus the compiled variant — the exact command is under Download.

aimodelc-h16c/ is that same model pre-compiled for one GPU architecture: identical output and speed, about half the cold-load time, and the runtime rejects it on a different architecture. Take it only if you are on h16c and care about load time.

What this is and is not

• It is a decode-optimized single-token-step graph, the shape a chat/completion loop uses.
• It is not a chunked-prefill or batch-serving asset.
• Quality here means teacher-forced top-1 agreement and cosine similarity against an independent fp32 Hugging Face reference, over 5 sequences / 125 positions. That is a regression probe, not a benchmark suite. No MMLU/GSM8K-style numbers are claimed.

Artifacts

Full SHA-256 of each file as published.

• The two tokenizer/tokenizer.json copies are byte-identical (same hash above). The one under aimodelc-h16c/tokenizer/ is a convenience copy; you do not need both.
• Portable and compiled are the same model, published together because the compiled one halves cold load but only runs on one architecture, so neither dominates.
• Both were produced from the same pinned commit and have identical compiled storage budgets.

Recipe, and what was rejected

Oracle for every quality number: an independent fp32 reference, Hugging Face's own Lfm2ForCausalLM at transformers ≥ 5.2, teacher-forced over 5 sequences / 125 positions. Cosine is the minimum per-position cosine.

The shipped arm has the highest minimum cosine of the int8 arms. The ±1 position differences between the three int8 rows are near-tie argmax flips at 125 positions, not a systematic ordering — do not read the embedding row's 123 as "better". int4 is excluded on quality, not size: a minimum cosine of 0.51–0.80 is a different model. Rejected variants are documented and deliberately not uploaded. Full detail in RECIPE.md.

Performance

Decode here is memory-bandwidth bound — throughput tracks bytes read per token.

Only 107.98 tok/s is attached to the published artifact in evidence/benchmark.json. The earlier 138.15 result was not reproduced after publication, so it is retained as historical context rather than advertised as this download's performance. The workload is memory-bandwidth-bound and was measured while the machine was busy; measure on your own hardware.

For reference, on the same Mac, MLX reported ~100 tok/s at 8-bit and ~60 tok/s at BF16. Those were throughput-only observations; MLX quality was not measured, so no quality comparison against MLX is claimed.

AOT compilation is throughput-neutral within noise and halves cold load, 9.8 s → 5.1 s.

Reproduce

Every pin, the full quantization spec, the four gates, the measurement protocol and the rejected variants are in RECIPE.md. Summary:

Source model : LiquidAI/LFM2.5-2.6B @ dca1825886789bd40b94368f53b1d9ada4c94598          (public)
Converter    : github.com/apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8  (public)
Runtime      : github.com/apple/coreai-models @ 5ed9981303b38d5a44aa6b45509bc4f6945029f5  (public)
Community LFM2 exporter, overlay and two runtime patches:
               github.com/john-rocky/coreai-model-zoo
               @ 95a29d41affed3bdf3ea5992ece094a908e21a04                     (public)
2.6B wrapper : lfm2.5-2.6b-coreai-conversion-e0e375b.tar.gz
               SHA-256 665285044be7c0171e1b737167b9892f2799bc2e01e3c5441212887903456f2d
Toolchain    : macOS 27.0 (26A5388g), Xcode 27.0 (27A5228h), Python 3.11.15,
               torch 2.9.0, coreai-core 1.0.0b2, coreai-torch 0.4.1,
               coreai-opt 0.2.1, coremltools 9.0

Verify a rebuild by the gates and the storage budget, not by hashing. The exporter names each externalized call site with a generated UUID (391 in this graph), so two exports of identical weights differ in a few bytes and therefore in SHA-256. The budget to match is Int8 2,621,243,392 + Float16 428,342,276 + Float32 34 + small index types.

Download

# The portable bundle: the .aimodel graph, the tokenizer and the bundle metadata.
# This is the whole repository except the compiled h16c variant (~3.5 GB rather than ~7 GB).
hf download harshav/LFM2.5-2.6B-CoreAI \
  --exclude 'aimodelc-h16c/*' \
  --local-dir ./LFM2.5-2.6B-CoreAI

# The directory you then point the runtime at is ./LFM2.5-2.6B-CoreAI itself.

With a runtime built as described above, the asset is driven as a pipelined Core AI language model with COREAI_CHUNK_THRESHOLD=1, greedy decoding, using the model's own chat template at tokenizer/chat_template.jinja. Operational notes:

COREAI_CHUNK_THRESHOLD=1 coreai-models/.build/release/llm-runner \
  --model ./LFM2.5-2.6B-CoreAI \
  --prompt "The capital of France is" \
  --max-tokens 64 --temperature 0 \
  --inference-engine-variant coreai-pipelined \
  --warmup exact --warmup-length 1

• macOS 27.0+ on Apple silicon. Validated on M4 Max.
• Runs on the GPU via an MPSGraph delegate. This is not an ANE asset: the KV dimension is dynamic, and --preferred-compute does not change the emitted delegate.
• Budget roughly bundle size plus KV cache, about 5 GB at 4096 context.
• aimodelc-h16c/ is architecture-locked to h16c; the runtime names the architecture it wanted when it refuses.

Validation evidence

Machine-readable under evidence/: authored_parity_vs_huggingface.json, fp32_reference.json, recipe_quality.json, conversion_gate.json, benchmark.json, compiled_storage_stats.json.

Four separate questions, not interchangeable:

• Authoring fidelity — re-authored module vs Hugging Face, fp32: 21/21 top-1, cosine 1.000000. This caught a real bug: the checkpoint sets rope_parameters.rope_theta = 1e7, and code reading only the legacy top-level key silently defaults to 1e6 — a 10× wrong RoPE that still produces fluent short text. Both the converter overlay and transformers 4.x hit it.
• Quantization damage — vs the fp32 oracle: 122/125, min cosine 0.997050.
• Conversion fidelity — bundle vs its own quantized weights run eagerly: 5/5 exact. Comparing to fp32 here would conflate quantization damage with conversion bugs.
• Throughput — see the caveat above.

Limitations and negative results

• Requires the two public community runtime patches listed above; it does not run on an unmodified apple/coreai-models runtime.
• No task-benchmark evaluation; quality is a 125-position regression probe.
• Multilingual support is inherited from upstream and was not re-verified per language; the probe is English.
• int4 rejected at block 32 (cosine 0.51–0.66) and at block 16 (quality recovers, 42 tok/s, ~3× slower than int8).
• --expect-frequent-reshapes measured 84 tok/s and 8.3 GB; not used.
• Speculative decoding not shipped: a static-S verify graph exports and its contract gates, but per-position logits do not match stepped decode.
• 300 tok/s was a target and was not reached by any tested configuration.

License and attribution

The model is Liquid AI's. This repository redistributes a converted, quantized copy under the upstream license, and claims no authorship of the model.

• Upstream: LiquidAI/LFM2.5-2.6B by Liquid AI.
• Upstream license: LFM Open License v1.0 — pinned copy here, included verbatim as LICENSE.upstream. Your use of these weights is governed by it.
• Official Apple tooling: apple/coreai-models, coreai-core, coreai-torch, and coreai-opt.
• Community conversion/runtime work: public john-rocky/coreai-model-zoo, maintained by Daisuke Majima, under its BSD-3-Clause license. Its repository includes third-party notices for portions derived from Apple's BSD-licensed coreai-models project.
• This repository contributes the conversion recipe, the gates, and the measurements.