LFM2.5-350M-OptiQ-4bit

mlx-community/LFM2.5-350M-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs

An OptiQ mixed-precision quant of LiquidAI/LFM2.5-350M, a 350M on-device model from Liquid AI. 269 MB on disk, down from 709 MB at bf16.

LFM2.5 is a hybrid architecture: 16 blocks alternating short convolutions with full attention (only 6 blocks carry a KV cache). OptiQ measures each layer's sensitivity and assigns per-layer bit-widths, so the layers that matter keep 8-bit while the rest go to 4-bit.

What it is

Capability Score

Six-metric mean, the standard OptiQ eval.

A 350M model sits near the floor on multi-step maths and long-context retrieval; those scores are genuine, not harness artifacts. Instruction-following and tool-calling are where a model this size is actually useful.

Run it

pip install mlx-optiq

from mlx_lm import load, generate

model, tok = load("mlx-community/LFM2.5-350M-OptiQ-4bit")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Name three colours."}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=200))

Serve it with the bundled mixed-precision KV cache:

optiq serve --model mlx-community/LFM2.5-350M-OptiQ-4bit --kv-config kv_config.json

Tool calling

LFM2.5 writes Pythonic calls between special tokens:

<|tool_call_start|>[get_weather(city="Paris")]<|tool_call_end|>

Pass tools= to apply_chat_template and they are rendered into the system prompt.

Links

• Project website: mlx-optiq.com
• All OptiQ quants: mlx-optiq.com/models
• Base model: LiquidAI/LFM2.5-350M