LFM2.5-8B-A1B-q4f16_1-MLC

LFM2.5-8B-A1B-q4f16_1-MLC

LiquidAI/LFM2.5-8B-A1B (Liquid AI), compiled to MLC / WebGPU (q4f16_1) for in-tab / on-device inference via WebLLM. Ported with Kiln, which adds the LFM2 hybrid architecture (LIV short-conv + GQA/QK-norm + DeepSeek-V3-style MoE) to mlc-llm — notably without any custom TVM operator (the conv-state rides an RNNState ring buffer alongside the KV cache).

• Quantization: q4f16_1 (~4.44 GB) · Context: 4096 · Decode: ~93 tok/s on an M4 Pro (Metal).
• Files: q4f16_1 weight shards + mlc-chat-config.json + tokenizer + LFM2.5-8B-A1B-q4f16_1-webgpu.wasm (WebGPU model lib).

Use (WebLLM)

import { CreateMLCEngine } from "@mlc-ai/web-llm";
const appConfig = { model_list: [{
  model: "https://huggingface.co/naklitechie/LFM2.5-8B-A1B-q4f16_1-MLC/resolve/main/",
  model_id: "LFM2.5-8B-A1B-q4f16_1-MLC",
  model_lib: "https://huggingface.co/naklitechie/LFM2.5-8B-A1B-q4f16_1-MLC/resolve/main/LFM2.5-8B-A1B-q4f16_1-webgpu.wasm",
  overrides: { context_window_size: 4096 },
}]};
const engine = await CreateMLCEngine("LFM2.5-8B-A1B-q4f16_1-MLC", { appConfig });

⚠️ Runtime match. Compiled with mlc-llm main (TVM 0.25.dev0); the cache manifest is tensor-cache.json. Load with a @mlc-ai/web-llm build whose bundled tvmjs matches — stock 0.2.84 expects an older runtime + ndarray-cache.json and will not load this as-is.

Note — reasoning model

LFM2.5-8B-A1B emits <think>…</think> reasoning. At q4 use sampling + a repetition penalty (defaults are baked into mlc-chat-config.json: temp 0.7 / top_p 0.95 / frequency_penalty 0.5) — greedy decoding degenerates into repetition (the unquantized original does too; this is model behavior, not a port artifact — see Kiln reference/gate-3-8b.md).

Attribution & license

Format conversion only. The original model and its LFM Open License are © Liquid AI (LiquidAI/LFM2.5-8B-A1B). MLC compilation: Kiln · NakliTechie/mlc-llm.