LFM2.5-350M-M3-Distill

LFM2.5-350M-M3-Distill

A 350M-parameter assistant fine-tuned from LiquidAI/LFM2.5-350M on a synthetic instruction dataset distilled from MiniMax-M3 (a frontier teacher model). It targets fast, capable on-device / Raspberry Pi use for everyday assistant tasks — general chat, Q&A, summarization, text rewriting, classification/extraction, and JSON / structured output.

v2 (current). Same recipe as v1 but trained on a verbosity-curated dataset — terse tasks (rewrite, "make concise", tone changes) now give a single clean answer instead of multi-option essays, with no regression on chat / Q&A / summarization / JSON / tool-calling. Read Limitations before using.

What it's good at

Versus the stock LFM2.5-350M base (observed on a fixed 35-prompt eval set), the fine-tune produces:

• More structured, complete answers for open-ended chat and explanations.
• Better short-text classification (e.g. spam / sentiment) — fixed cases the base got wrong.
• Cleaner summarization and JSON output for its size.
• Tool / function calling is preserved from the base (native <|tool_call_start|> format) — and on math it correctly defers to a calculate tool instead of hallucinating arithmetic.

Intended use

• General chat & advice
• Factual Q&A
• Summarization
• Text rewriting (tone, formality, paraphrase, simple translation)
• Classification & extraction (sentiment, spam, fields → JSON)
• Structured / JSON output
• Tool / function calling (native LFM2.5 format)

Limitations (please read)

• Not for code or math. The LFM2.5 base is explicitly not recommended for code, math, or creative writing, and this model was deliberately not trained on the code/reasoning data. Use a code-capable model (e.g. Qwen3-0.6B) for those.
• Weak at precise token-level editing. Grammar/spelling-correction tasks are unreliable — a genuine capacity limit at 350M (unchanged in v2).
• Shallow synthesis. When summarizing long/multi-source material it tends to list findings rather than deeply synthesize — expected at this size.
• Small model: it can hallucinate and should not be relied on for factual accuracy without verification.

Recommended generation settings

Use sampling (not greedy) — greedy decoding causes repetition loops at this size:

generation_config = dict(
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.1,
    max_new_tokens=512,
)

Usage (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("aruntemme/LFM2.5-350M-M3-Distill")
model = AutoModelForCausalLM.from_pretrained("aruntemme/LFM2.5-350M-M3-Distill",
                                             dtype=torch.bfloat16).to("cuda")

msgs = [{"role": "user", "content": "Summarize the benefits of regular exercise in 3 bullet points."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
                                 return_dict=True).to("cuda")
out = model.generate(**inputs, do_sample=True, temperature=0.7, top_p=0.9,
                     repetition_penalty=1.1, max_new_tokens=512)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Tool / function calling

Pass tools via the chat template; the model emits calls in LFM2.5's native format (<|tool_call_start|>[func(arg="val")]<|tool_call_end|>):

tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Get current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
msgs = [{"role": "user", "content": "What's the weather in Paris?"}]
inputs = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                                 return_tensors="pt", return_dict=True).to("cuda")
out = model.generate(**inputs, max_new_tokens=100)
# -> <|tool_call_start|>[get_weather(city="Paris")]<|tool_call_end|>

Usage (GGUF / llama.cpp / Raspberry Pi)

Quantized GGUF files are included for CPU / edge inference:

./llama-cli -m lfm2.5-350m-m3distill-Q4_K_M.gguf \
  -p "Explain photosynthesis in 40 words." \
  --temp 0.7 --top-p 0.9 --repeat-penalty 1.1

Training

• Base: LiquidAI/LFM2.5-350M
• Teacher: MiniMax-M3 (synthetic data generated via the teacher, then quality-filtered/judged)
• Method: full supervised fine-tuning (no LoRA), bf16, gradient checkpointing
• Data (v2): ~30,400 instruction→response examples (chat, Q&A, summarization, rewriting, explanation, classification/extraction, JSON) — code & reasoning excluded to match the base's strengths; v2 additionally drops the multi-option / over-long responses on terse tasks to cut needless verbosity (v1 used ~32,100 uncurated examples)
• Hyperparameters: 2 epochs, effective batch 32 (bs 4 × grad-accum 8), lr 1.5e-5 (cosine, 3% warmup), max sequence length 1024, weight decay 0.01
• Hardware: single NVIDIA RTX 3060 (12 GB), ~2.7 h
• Final eval: loss 1.87, token accuracy 0.60
• Frameworks: TRL 1.6.0, Transformers 5.12.1, PyTorch 2.12.1

License & attribution

This is a derivative of LiquidAI/LFM2.5-350M, distributed under the LFM Open License v1.0 (commercial use permitted under a $10M annual-revenue threshold). This derivative is provided under the same license; a copy is included. The base model and its license notices are retained per the license terms.

Acknowledgements

• Base model: Liquid AI (LFM2.5-350M)
• Distillation teacher: MiniMax-M3