LFM2.5-350M-Thinking-GGUF

<div align="center"> <img src="https://huggingface.co/KoarAI/LFM2.5-350M-Thinking-GGUF/resolve/main/gguf_banner.png" alt="KoarAI LFM2.5-350M Thinking GGUF Banner" width="100%" style="border-radius: 12px; box-shadow: 0 4px 20px rgba(0,0,0,0.3);"/>

🐨 KoarAI / LFM2.5-350M-Thinking-GGUF

</div>

🌟 Overview

This repository contains official GGUF format quantizations of KoarAI/LFM2.5-350M-Thinking, an ultra-lightweight reasoning model trained with Full Fine-Tuning on multi-teacher reasoning distillation traces (Qwen 3.8 Max, GLM 5.2, and Kimi K3).

These GGUF files are ready for high-speed inference across llama.cpp, Ollama, LM Studio, Jan.ai, and mobile/edge devices.

---

📦 Available Quantizations

---

⚡ Quickstart with Ollama

Create a Modelfile:

FROM ./LFM2.5-350M-Thinking-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.9

Run in terminal:

ollama create koarai-350m -f Modelfile
ollama run koarai-350m "How many 'r' in strawberry?"

---

💻 Quickstart with llama.cpp

./llama-cli -m ./LFM2.5-350M-Thinking-Q4_K_M.gguf \
  -p "<|im_start|>user\nSolve: 2x + 10 = 24<|im_end|>\n<|im_start|>assistant\n<think>\n" \
  -n 512 --temp 0.6

---

🐨 Maintained by KoarAI Lab

Released for the open-source AI community by KoarAI.