LFM2.5-8B-A1B

LFM2.5-8B-A1B for hipfire

Pre-quantized LFM2.5-8B-A1B for hipfire, a Rust-native LLM inference engine for AMD RDNA GPUs (HIP/ROCm-direct, no Python in the hot path).

Quantized from LiquidAI/LFM2.5-8B-A1B — Liquid AI's reasoning-tuned MoE hybrid: 8.3B total / 1.5B active parameters, 24 layers (18 double-gated LIV convolution blocks + 6 GQA blocks), 128,000-token context, vocab 128,000.

These .mq* files are hipfire's HFQ container format — not GGUF or safetensors; they won't load in llama.cpp / transformers.

Files

The upstream LiquidAI chat_template.jinja is embedded in the file's HFQ metadata; hipfire's jinja chat path picks it up by default. The model is reasoning-tuned and emits <think> blocks before answering.

Usage

# Install hipfire
curl -L https://raw.githubusercontent.com/Kaden-Schutt/hipfire/master/scripts/install.sh | bash

# Pull and run via the registry
hipfire pull lfm2.5:8b-a1b
hipfire run lfm2.5:8b-a1b "What is the capital of France?"

# Or download the file directly and serve it
hf download hipfire-models/hipfire-LFM2.5-8B-A1B lfm2.5-8b-a1b.mq4 --local-dir ~/.hipfire/models
hipfire serve --model lfm2.5-8b-a1b.mq4

Recommended sampling (from the upstream card): temperature 0.2, top_k 80, repetition_penalty 1.05.

Quantization & validation

MQ4 (MagnumQuant 4-bit) — FWHT-rotated 4-bit: weights are pre-rotated through a Walsh–Hadamard transform at quantization time and the input vector is rotated through the same transform on the fly inside the GEMV kernels. The rotation flattens outliers, giving roughly Q8-grade output at 4-bit storage.

This exact file was validated on gfx1100 (RX 7900 XTX): daemon load + greedy generation, full coherence-detector battery clean (no attractors, no special-token leaks, no loops), ≈300 tok/s decode on an 80-token greedy probe.

License & attribution

This is a quantized Derivative Work of LiquidAI/LFM2.5-8B-A1B, original work copyright Liquid AI, Inc., redistributed under the LFM Open License v1.0 (the LICENSE file in this repo is copied verbatim from the upstream repository).

Modification notice (License §4(b)): the upstream safetensors weights were re-quantized into hipfire's MQ4/HFQ container format, and the upstream chat_template.jinja was embedded into the HFQ metadata (the export pipeline did not carry it). No other changes.

Note the license's Commercial Use limitation (§5): commercial use is licensed only for entities below US$10M annual revenue — read the LICENSE.