LFM2.5-2.6B-GGUF

LFM2.5-2.6B GGUF

GGUF quantized versions of LiquidAI/LFM2.5-2.6B, a high-performance hybrid model designed for on-device deployment, featuring a 128K context window and advanced agentic capabilities.

Model Overview

LFM2.5-2.6B is part of the LFM2.5 family, building on the LFM2 architecture to provide best-in-class performance for its size. It is specifically optimized for agentic workloads, tool use, and long-context workflows, offering competitive performance against models 4x its size.

Key features include:

• Agentic Post-Training: Trained using agentic reinforcement learning for improved tool use and instruction following.
• Efficient Inference: Designed for high-speed execution on both CPU and GPU.
• Reasoning Capabilities: A pure reasoning model that utilizes a <think> tag to reason before answering.
• Massive Context: Supports up to 131,072 tokens.

Model Architecture

Available GGUF Files

Recommended: Q4_K_M or Q5_K_M offer the best quality-to-size trade-off for most use cases.

Usage

llama.cpp CLI

./llama-cli \
  -m lfm2.5-2.6b-Q4_K_M.gguf \
  -p "What is the capital of France?" \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

llama-server (OpenAI-compatible API)

./llama-server \
  -m lfm2.5-2.6b-Q4_K_M.gguf \
  --host 0.0.0.0 --port 8080

Chat Template & Reasoning

LFM2.5 uses a ChatML-like format. It is a reasoning model that automatically adds a <think> tag when starting an assistant answer to process its logic before providing the final response.

Example format:

<|startoftext|><|im_start|>system
You are a helpful assistant trained by Liquid AI.<|im_end|>
<|im_start|>user
What is C. elegans?<|im_end|>
<|im_start|>assistant
<think>
... reasoning process ...
</think>
C. elegans is a species of small roundworm...<|im_end|>

Tool Calling

LFM2.5 supports Pythonic function calling. It outputs function calls between <|tool_call_start|> and <|tool_call_end|> tokens.

Generation Parameters

Recommended parameters for optimal performance:

Quantization

These GGUF files were created using llama.cpp tools to enable efficient local deployment on CPUs and GPUs with reduced memory footprints.

Acknowledgements

• Original model: LiquidAI/LFM2.5-2.6B
• Quantization tool: llama.cpp

License

LFM 1.0 License