LFM2.5-350M

LFM2.5-350M for hipfire

Pre-quantized LFM2.5-350M for hipfire, a Rust-native LLM inference engine for AMD RDNA GPUs (HIP/ROCm-direct, no Python in the hot path).

Quantized from LiquidAI/LFM2.5-350M — Liquid AI's 350M-parameter LFM2.5 hybrid: 16 layers (10 double-gated LIV convolution blocks + 6 GQA blocks), 32,768-token context, vocab 65,536. Liquid recommends it for data extraction, structured outputs, and tool use.

These .mq* files are hipfire's HFQ container format — not GGUF or safetensors; they won't load in llama.cpp / transformers.

Files

The upstream LiquidAI chat_template.jinja is embedded in the file; hipfire's jinja chat path picks it up by default.

Usage

# Install hipfire
curl -L https://raw.githubusercontent.com/Kaden-Schutt/hipfire/master/scripts/install.sh | bash

# Pull and run via the registry
hipfire pull lfm2.5:350m
hipfire run lfm2.5:350m "Extract the date from: meeting moved to March 3rd"

# Or download the file directly and serve it
hf download hipfire-models/hipfire-LFM2.5-350M lfm2.5-350m.mq4 --local-dir ~/.hipfire/models
hipfire serve --model lfm2.5-350m.mq4

Recommended sampling (from the upstream card): temperature 0.1, top_k 50, repetition_penalty 1.05.

Quantization

MQ4 (MagnumQuant 4-bit) — FWHT-rotated 4-bit: weights are pre-rotated through a Walsh–Hadamard transform at quantization time and the input vector is rotated through the same transform on the fly inside the GEMV kernels. The rotation flattens outliers, giving roughly Q8-grade output at 4-bit storage. Validated coherent on gfx1100 (RX 7900 XTX).

License & attribution

This is a quantized Derivative Work of LiquidAI/LFM2.5-350M, original work copyright Liquid AI, Inc., redistributed under the LFM Open License v1.0 (the LICENSE file in this repo is copied verbatim from the upstream repository).

Modification notice (License §4(b)): the upstream safetensors weights were re-quantized into hipfire's MQ4/HFQ container format. No other changes.

Note the license's Commercial Use limitation (§5): commercial use is licensed only for entities below US$10M annual revenue — read the LICENSE.