LFM2.5-1.2B-JP-202606-OptiQ-4bit

LFM2.5-1.2B-JP-202606-OptiQ-4bit

Built with mlx-optiq, an MLX-native toolkit for quantizing, fine-tuning, and serving language models on Apple Silicon.

Documentation: https://mlx-optiq.com/docs
Model collection: https://mlx-optiq.com/models

mlx-community/LFM2.5-1.2B-JP-202606-OptiQ-4bit is a mixed-precision MLX quantization of LiquidAI/LFM2.5-1.2B-JP-202606.

The model occupies 838 MB on disk compared to 2.34 GB for the original bf16 release. According to the included evaluation, it achieves 58.5% pass@1 on HumanEval.

LFM2.5 uses a hybrid architecture consisting of convolutional layers interleaved with full-attention layers. Only the attention layers maintain a KV cache. OptiQ performs mixed-precision quantization by assigning different bit widths to individual layers based on measured sensitivity.

---

Model Summary

---

Evaluation

The following metrics are included with this release.

The Capability Score is the arithmetic mean of the six benchmark scores reported above.

---

Usage

Install the required package:

pip install mlx-optiq

Load the model:

from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/LFM2.5-1.2B-JP-202606-OptiQ-4bit"
)

prompt = tokenizer.apply_chat_template(
    [
        {
            "role": "user",
            "content": "日本の首都はどこですか？"
        }
    ],
    tokenize=False,
    add_generation_prompt=True,
)

print(
    generate(
        model,
        tokenizer,
        prompt=prompt,
        max_tokens=300,
    )
)

To serve the model with the bundled mixed-precision KV cache:

optiq serve \
    --model mlx-community/LFM2.5-1.2B-JP-202606-OptiQ-4bit \
    --kv-config kv_config.json

---

Tool Calling

The model emits tool calls using the following format:

<|tool_call_start|>[get_weather(city="Tokyo")]<|tool_call_end|>

When using apply_chat_template, tool definitions can be supplied through the tools= argument.

---

Notes

• This repository contains a mixed-precision MLX quantization of the original model.
• Quantization was performed using OptiQ.
• The included kv_config.json contains the KV-cache precision configuration used by this release.
• Benchmark results reported above are those included with this release.

---

Resources

• Base model: https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP-202606
• mlx-optiq: https://mlx-optiq.com
• Documentation: https://mlx-optiq.com/docs
• Model collection: https://mlx-optiq.com/models