LFM2.5-2.6B-GGUF

LFM2.5-2.6B

LFM2.5-2.6B is a compact general-purpose text model developed by Liquid AI as part of the LFM2.5 family of hybrid models designed for efficient on-device deployment. The model combines a lightweight architecture with agentic post-training, a 128K context window, and capabilities for tool use, instruction following, multi-step agentic workflows, data extraction, and retrieval-augmented generation (RAG).

This repository contains GGUF quantized variants of LFM2.5-2.6B optimized for efficient local inference using llama.cpp. The quantized formats substantially reduce the model's memory footprint while retaining its core language understanding and agentic capabilities.

LFM2.5-2.6B is a text-only model, not a vision-language or multimodal model. Its primary focus is efficient agentic language processing, particularly for applications where low latency, long context, tool interaction, and local execution are important.

---

Model Overview

• Model Name: LFM2.5-2.6B
• Base Model: LiquidAI/LFM2.5-2.6B
• Architecture: Hybrid LFM2 architecture with short-convolution blocks and grouped-query attention
• Parameter Count: 2.69 Billion Parameters
• Modalities: Text
• Context Length: 131,072 tokens
• Vocabulary Size: 128,000
• Languages: Multilingual
• Developer: Liquid AI
• License: LFM1.0

---

Quantization Formats

This repository provides GGUF quantized versions of LFM2.5-2.6B optimized for efficient local inference using llama.cpp.

IQ3_M

• Size reduction of approx 76.14% (1.20 GB) compared to 16-bit (5.03 GB)
• Aggressive 3-bit quantization designed for highly memory-efficient deployment of the 2.6B model
• Suitable for edge AI, local agent workloads, and systems where available memory is limited
• Enables practical execution of long-context language workloads with a substantially smaller model footprint
• More demanding reasoning, tool-use sequences, and complex multi-step interactions may experience greater quality degradation than higher-precision variants

IQ4_NL

• Size reduction of approx 70.38% (1.49 GB) compared to 16-bit (5.03 GB)
• Advanced 4-bit non-linear quantization providing a stronger balance between model fidelity and memory reduction
• Well suited for agentic workflows, tool interaction, RAG, structured information extraction, and long-context processing
• Provides improved retention of language and reasoning behavior compared with more aggressive 3-bit compression
• May require additional computational resources compared with the smaller IQ3_M variant

IQ4_XS

• Size reduction of approx 71.77% (1.42 GB) compared to 16-bit (5.03 GB)
• Compact 4-bit quantization offering a balanced deployment profile for local and edge inference
• Suitable for conversational workloads, multi-step agent tasks, retrieval-based applications, and structured data extraction
• Provides a practical compromise between model footprint and generation quality
• Appropriate for deployments where memory efficiency and stable agentic behavior are both important

---

Training Background (Original Model)

LFM2.5-2.6B is built on the LFM2 hybrid architecture and was pretrained on approximately 34 trillion tokens. Its mid-training process extended the model's context capability to 128K tokens. The post-training pipeline subsequently transformed the base model into an agent-oriented model.

Pretraining

• Approximately 34T training tokens used during pretraining
• Hybrid architecture combining double-gated short-convolution blocks with grouped-query attention
• Mid-training stage used to extend context capability to 128K tokens
• Designed for efficient language processing and deployment across constrained hardware environments

Agentic Post-Training

The post-training process consists of multiple stages:

• Supervised fine-tuning performed in two rounds
• Per-domain teacher specialization
• Multi-domain on-policy distillation
• Agentic reinforcement learning using popular agentic harnesses

The agentic reinforcement-learning stage exposes the model to tools, system prompts, and interaction patterns used by agent environments, improving its ability to operate through multi-step tool-based workflows.

---

Key Capabilities

• Agentic Task Execution Designed to work through multi-step tasks rather than limiting interaction to single-turn text generation.

• Tool Use & Function Calling Supports tool definitions, function calls, tool-result interpretation, and final response generation within an agent workflow.

• Long-Context Processing Supports a context length of up to 131,072 tokens, making it suitable for applications involving large inputs and extended interactions.

• Data Extraction Suitable for extracting structured information from unstructured textual inputs.

• Retrieval-Augmented Generation Recommended by Liquid AI for RAG-oriented applications where retrieved information must be incorporated into multi-step language workflows.

• Instruction Following Post-training emphasizes reliable adherence to instructions across conversational and agentic workflows.

• Multilingual Processing Supports 16 languages spanning European, Asian, and other language groups.

• Efficient Local Inference The LFM2 architecture is specifically designed for efficient deployment across CPU, GPU, and other constrained hardware environments. Liquid AI reports 220 tokens/s on an Apple M5 Max and 113 tokens/s on an AMD Ryzen AI Max+ 395 for the original model configuration.

---

Recommended Usecases

• Agentic AI Systems Build autonomous or semi-autonomous systems capable of performing multi-step tasks.

• Tool-Calling Applications Integrate the model with external functions, APIs, databases, and application tools.

• RAG Systems Build retrieval-augmented assistants capable of processing retrieved knowledge within long contexts.

• Structured Data Extraction Extract organized information from large volumes of unstructured text.

• Long-Context Applications Process large documents and extended conversational contexts within a 128K-token context window.

• On-Device AI Deploy language intelligence locally on phones, laptops, edge systems, and other constrained devices.

• Conversational Assistants Build lightweight assistants requiring low-latency local inference.

Limitations

LFM2.5-2.6B is primarily optimized for agentic workloads, tool use, data extraction, RAG, and long-context applications. The official model card specifically states that it is not recommended for agentic coding or knowledge-heavy tasks.

---

Usage Example

Using llama.cpp

./llama-cli \
  -m SandLogicTechnologies/LFM2.5-2.6B-IQ4_NL.gguf \
  -p "Analyze the provided information, identify the important entities, and return the results as structured JSON."

---

Acknowledgments

These quantized models are based on the original work by the Liquid AI development team.

Special thanks to:

• The Liquid AI team for developing and releasing LFM2.5-2.6B.
• The developers and open-source community behind llama.cpp for enabling efficient GGUF inference across CPU, GPU, and edge hardware.

---

Contact

For questions, feedback, or support, please reach out at [email protected] or visit https://www.sandlogic.com/