LFM2.5-2.6B-FP8

LFM2.5-2.6B-FP8

Quantized build of LiquidAI/LFM2.5-2.6B, produced by vrfai.

Format: FP8 (W8A8), SmoothQuant + static per-tensor activation scales from calibration, serialized as compressed-tensors. Size: 2.77 GB (base model: 5.0 GB, 55% of original).

What was quantized

LFM2.5-2.6B is a hybrid stack: 30 Lfm2DecoderLayers, 8 full_attention and 22 conv (Lfm2ShortConv — a depthwise causal nn.Conv1d, kernel size 3, not an SSM with a learned decay gate the way Gated DeltaNet is, so unlike that architecture there is no fragile recurrence-decay sub-component inside the conv block worth protecting). Only lm_head is held at bf16; every other Linear quantizes — 1 module(s) excluded (lm_head).

90.28% of all nn.Linear weight (by parameter count) is quantized — see the source pipeline's inspect_model.py for the full per-role census.

Quantization configuration

Benchmark results

Two tasks, chosen because they exist in the installed lm_eval harness and are the closest deterministic (no LLM-judge) match to what LiquidAI's own model card reports for LFM2.5-2.6B: IFBench/Multi-IF (instruction-following) → ifeval; AIME25 (math reasoning) → aime25, a literal name match to the card's own reported score.

LFM2.5's chat template unconditionally opens every generation with <think> (no non-thinking toggle exists — verified directly against chat_template.jinja), so both tasks run in a _think variant that strips everything up to </think> before scoring, and a mild repetition_penalty=1.1 (fully deterministic, no sampling) is applied — without it, plain greedy decoding measurably got stuck in degenerate token-repeat loops on ifeval for both checkpoints, inflating truncated responses with a cause that has nothing to do with quantization.

Both deltas are within run-to-run noise at these sample sizes (two-proportion z-test, p > 0.05) — no measurable regression from FP8 quantization on either task. AIME25 (n=30) has limited statistical power by nature of the dataset size; IFEval (n=541) is the more reliable signal of the two and shows FP8 matching or slightly exceeding the bf16 baseline.

Usage

No quantization flag is required — vLLM reads the format from config.json.

vllm serve vrfai/LFM2.5-2.6B-FP8 --trust-remote-code

from vllm import LLM, SamplingParams

llm = LLM(model="vrfai/LFM2.5-2.6B-FP8", trust_remote_code=True)
print(llm.generate(["The capital of France is"],
                   SamplingParams(temperature=0.0, max_tokens=64))[0].outputs[0].text)

---

[!WARNING] Everything below this line is reproduced verbatim from the base model card, LiquidAI/LFM2.5-2.6B. The benchmark tables it contains (AA-Omni, AIME25, LiveCodeBench, IFBench, BFCL, ...) were measured by LiquidAI on the original weights. They were not measured on this quantized checkpoint. For numbers measured on these weights, see the Benchmark results section above.

---

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" /> <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;"> <a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> • <a href="https://docs.liquid.ai/lfm/getting-started/welcome"><strong>Docs</strong></a> • <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a> • <a href="https://discord.com/invite/liquid-ai"><strong>Discord</strong></a> </div> </div>

LFM2.5-2.6B

LFM2.5-2.6B is part of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with a 128K context window and agentic post-training.

• Best-in-class agent: Competitive with models 4x larger on tool use, instruction following, and multi-step agentic tasks.
• Agentic reinforcement learning: Trained inside the most popular agentic harnesses to improve compatibility.
• Efficient inference: 220 tok/s on an Apple M5 Max and 113 tok/s on an AMD Ryzen CPU, in under 2.5 GB of memory.

Find more information about LFM2.5-2.6B in our blog post.

[!NOTE] 💻 Demos: Try LFM2.5-2.6B's agentic capabilities in a Hugging Face space without any setup: Research Agent in your browser: helps you research a specific question and generates a summary

🗒️ Model Details

LFM2.5-2.6B is a general-purpose text-only model with the following features:

• Total parameters: 2.69B
• Number of layers: 30 (22 double-gated short convolution blocks + 8 GQA)
• Training budget: 34 trillion tokens
• Vocabulary size: 128,000
• Context length: 131,072 tokens
• Languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish
• Generation parameters:
  • temperature: 0.1
  • top_k: 50
  • repetition_penalty: 1.1

We recommend using it for agentic workloads, tool use, data extraction, RAG, and long-context workflows. It is not recommended for agentic coding and knowledge-heavy tasks.

Chat Template

LFM2.5 uses a ChatML-like format. See the Chat Template documentation for details. Example:

<|startoftext|><|im_start|>system
You are a helpful assistant trained by Liquid AI.<|im_end|>
<|im_start|>user
What is C. elegans?<|im_end|>
<|im_start|>assistant

You can use tokenizer.apply_chat_template() to format your messages automatically.

[!TIP] 💡 Note: LFM2.5-2.6B is a pure reasoning model that always thinks before it answers. It adds a <think> tag directly in the chat template when starting an assistant answer.

Tool Use

LFM2.5 supports function calling in four steps:

1. Function definition: Provide the list of tools as a JSON object in the system prompt, or use tokenizer.apply_chat_template() with tools=....
2. Function call: By default, LFM2.5 writes Pythonic function calls (a Python list between <|tool_call_start|> and <|tool_call_end|> special tokens), as the assistant answer. You can override this behavior by asking the model to output JSON function calls in the system prompt.
3. Function execution: Execute the call and return the result with the tool role.
4. Final answer: LFM2.5 interprets the tool output and returns a plain-text answer addressing the original prompt.

See the Tool Use documentation for the full guide. Example:

<|startoftext|><|im_start|>system
List of tools: [{"name": "get_candidate_status", "description": "Retrieves the current status of a candidate in the recruitment process", "parameters": {"type": "object", "properties": {"candidate_id": {"type": "string", "description": "Unique identifier for the candidate"}}, "required": ["candidate_id"]}}]<|im_end|>
<|im_start|>user
What is the current status of candidate ID 12345?<|im_end|>
<|im_start|>assistant
<|tool_call_start|>[get_candidate_status(candidate_id="12345")]<|tool_call_end|>Checking the current status of candidate ID 12345.<|im_end|>
<|im_start|>tool
[{"candidate_id": "12345", "status": "Interview Scheduled", "position": "Clinical Research Associate", "date": "2023-11-20"}]<|im_end|>
<|im_start|>assistant
The candidate with ID 12345 is currently in the "Interview Scheduled" stage for the position of Clinical Research Associate, with an interview date set for 2023-11-20.<|im_end|>

Training

LFM2.5-2.6B is pre-trained on ~34T tokens, with a mid-training phase that extends the context window to 128K. Post-training then turns the base model into an agent in four stages: supervised fine-tuning (two rounds), per-domain teacher specialization, multi-domain on-policy distillation, and agentic reinforcement learning.

In particular, agentic reinforcement learning allows us to directly train the model inside popular agentic harnesses. It exposes the model to their tools, system prompts, and interaction patterns, helping it work reliably across agent environments.

🏃 Inference

LFM2.5 is supported by many inference frameworks. See the Inference documentation for the full list.

[!TIP] ⚡ Faster decoding: attach LFM2.5-2.6B-DSpark, a 328M speculative-decoding drafter, for ~2.6x faster decoding in SGLang and on Apple silicon via Metal with exactly the same outputs.

How to use

LFM2.5-2.6B can be used for direct inference or as a backend for agentic workflows.

Quick start

Get started with Transformers (compatible with transformers>=5.0.0):

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

model_id = "LiquidAI/LFM2.5-2.6B"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="bfloat16",
#   attn_implementation="flash_attention_2" <- uncomment on compatible GPU
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

prompt = "What is C. elegans?"

input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    return_tensors="pt",
    tokenize=True,
)["input_ids"].to(model.device)

output = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.1,
    top_k=50,
    repetition_penalty=1.1,
    max_new_tokens=512,
    streamer=streamer,
)

Agent Use

LFM2.5-2.6B supports tool calling for agentic workflows. Serve it locally with any OpenAI-compatible backend (see 🏃 Inference, then configure your agent harness to connect to it. For full setup instructions including installation and additional options, see our Agent Harnesses guide.

Note: The port depends on your serving backend — llama.cpp and MLX use 8080, vLLM uses 8000, SGLang uses 30000, and LM Studio uses 1234. Adjust the URLs below accordingly.

Hermes

Either use the interactive wizard or set it directly:

hermes config set model.provider custom
hermes config set model.base_url http://localhost:8080/v1
hermes config set model.default LFM2.5-2.6B
hermes config set model.context_length 131072
hermes config set model.api_mode chat_completions
hermes config set agent.tool_use_enforcement true

OpenClaw

Add to your config to models.providers:

local: {
  baseUrl: "http://localhost:8080/v1",
  apiKey: "sk-local",
  api: "openai-completions",
  models: [{
    id: "LFM2.5-2.6B",
    name: "LFM2.5-2.6B",
    contextWindow: 131072,
    maxTokens: 8192,
    cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }
  }]
}

Pi

Add to your config to ~/.pi/agent/models.json:

{
  "providers": {
    "local": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "local",
      "models": [{ "id": "LFM2.5-2.6B" }]
    }
  }
}

🔧 Fine-Tuning

We recommend fine-tuning LFM2.5 for your specific use case to achieve the best results.

📊 Performance

Benchmarks

We compared LFM2.5-2.6B with relevant sub-10B models on a diverse suite of benchmarks.

CPU Inference

Due to its efficient LFM2 architecture, LFM2.5-2.6B is the fastest model we tested, with decode speeds of 220 tokens/s on an M5 Max and 113 tokens/s on a Ryzen AI Max+ 395. At 30 tokens/s, it allows you to run capable agents even on a phone.

GPU Inference

LFM2.5-2.6B is the fastest model in its size class, reaching almost 15K output tokens per second at high concurrency, roughly 1.3B tokens per day on a single H100.

📬 Contact

• Got questions or want to connect? Join our Discord community
• If you are interested in custom solutions with edge deployment, please contact our sales team.

Citation

@article{liquidAI202626B,
  author  = {Liquid AI},
  title   = {LFM2.5-2.6B: Agents Everywhere},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {www.liquid.ai/blog/lfm2-5-2-6b},
}

@article{liquidai2025lfm2,
  title   = {LFM2 Technical Report},
  author  = {Liquid AI},
  journal = {arXiv preprint arXiv:2511.23404},
  year    = {2025}
}