LFM2.5-8B-A1B-AWQ-FP8

<div align="left"> <a href="https://cyan.kiwi" target="_blank"><img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-banners-cyankiwi.png" width="800" alt="cyankiwi, the LLM inference agent for open models"></a> </div> <div align="left"> <table style="border-collapse:collapse; border:none;"> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Website</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://cyan.kiwi/" target="_blank">cyan.kiwi</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Models</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://huggingface.co/cyankiwi" target="_blank">Hugging Face</a> · <a href="https://www.modelscope.ai/organization/cyankiwi" target="_blank">ModelScope</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Connect</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://x.com/cyankiwi_ai" target="_blank">X</a> · <a href="https://www.reddit.com/r/cyankiwi/" target="_blank">Reddit</a> · <a href="mailto:[email protected]">Email</a></td> </tr> </table> </div> <br> <div align="left"> <img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-banners-awq.png" width="800" alt="cyankiwi Activation-aware Weight Quantization"> </div> <div align="left"> <table style="border-collapse:collapse; border:none;"> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Version</b></td> <td align="left" style="border:none; padding:4px 0;">cyankiwi 26.05.01</td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Calibration</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://huggingface.co/datasets/cyankiwi/calibration" target="_blank">STEM and Agentic</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Model Size</b></td> <td align="left" style="border:none; padding:4px 0;">9.13 GB</td> </tr> </table> </div> <br> <p align="left"> <a href="https://cyan.kiwi/workspace?prompt=Deploy%20cyankiwi/LFM2.5-8B-A1B-AWQ-FP8" target="_blank"><img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-button-deploy.png" width="333" alt="Deploy on cyan.kiwi private endpoints"></a> </p>

---

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" /> <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;"> <a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> • <a href="https://docs.liquid.ai/lfm/getting-started/welcome"><strong>Docs</strong></a> • <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a> • <a href="https://discord.com/invite/liquid-ai"><strong>Discord</strong></a> </div> </div>

LFM2.5-8B-A1B

LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.

• On-device personal assistant: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
• Compressed performance: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
• Unmatched throughput: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.

Find more information about LFM2.5-8B-A1B in our blog post.

*AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.

🗒️ Model Details

LFM2.5-8B-A1B is a general-purpose text-only model with the following features:

• Total parameters: 8.3B
• Active parameters: 1.5B
• Number of layers: 24 (18 double-gated conv + 6 GQA)
• Training budget: 38 trillion tokens
• Context length: 128,000
• Vocabulary size: 128,000
• Languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish
• Generation parameters: We recommend the following parameters:
  • temperature: 0.2
  • top_k: 80
  • repetition_penalty: 1.05

We recommend using LFM2.5-8B-A1B for agentic workflows, tool use, structured outputs, multilingual assistants, and on-device personal-assistant applications. It is not the best fit for heavy programming or knowledge-intensive question answering without retrieval.

Chat Template

LFM2.5 uses a ChatML-like format. See the Chat Template documentation for details. Example:

<|startoftext|><|im_start|>system
You are a helpful assistant trained by Liquid AI.<|im_end|>
<|im_start|>user
What is C. elegans?<|im_end|>
<|im_start|>assistant

Because LFM2.5-8B-A1B is a reasoning model, assistant turns contain an explicit chain of thought before the final answer. You can use tokenizer.apply_chat_template() to format your messages automatically.

Tool Use

LFM2.5 supports function calling in four steps:

1. Function definition: Provide the list of tools as a JSON object in the system prompt, or use tokenizer.apply_chat_template() with tools=....
2. Function call: By default, LFM2.5 writes Pythonic function calls (a Python list between <|tool_call_start|> and <|tool_call_end|> special tokens), as the assistant answer. You can override this behavior by asking the model to output JSON function calls in the system prompt.
3. Function execution: Execute the call and return the result with the tool role.
4. Final answer: LFM2.5 interprets the tool output and returns a plain-text answer addressing the original prompt.

See the Tool Use documentation for the full guide. Example:

<|startoftext|><|im_start|>system
List of tools: [{"name": "get_candidate_status", "description": "Retrieves the current status of a candidate in the recruitment process", "parameters": {"type": "object", "properties": {"candidate_id": {"type": "string", "description": "Unique identifier for the candidate"}}, "required": ["candidate_id"]}}]<|im_end|>
<|im_start|>user
What is the current status of candidate ID 12345?<|im_end|>
<|im_start|>assistant
<|tool_call_start|>[get_candidate_status(candidate_id="12345")]<|tool_call_end|>Checking the current status of candidate ID 12345.<|im_end|>
<|im_start|>tool
[{"candidate_id": "12345", "status": "Interview Scheduled", "position": "Clinical Research Associate", "date": "2023-11-20"}]<|im_end|>
<|im_start|>assistant
The candidate with ID 12345 is currently in the "Interview Scheduled" stage for the position of Clinical Research Associate, with an interview date set for 2023-11-20.<|im_end|>

🏃 Inference

LFM2.5-8B-A1B is supported by many inference frameworks. See the Inference documentation for the full list.

[!TIP] ⚡ Faster decoding: attach LFM2.5-8B-A1B-DSpark, a 328M speculative-decoding drafter, for ~2.5x faster decoding in SGLang with exactly the same outputs.

Quick start with Transformers (compatible with transformers>=5.0.0):

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

model_id = "LiquidAI/LFM2.5-8B-A1B"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="bfloat16",
#   attn_implementation="flash_attention_2" <- uncomment on compatible GPU
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

prompt = "What is C. elegans?"

input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    return_tensors="pt",
    tokenize=True,
)["input_ids"].to(model.device)

output = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.2,
    top_k=80,
    repetition_penalty=1.05,
    max_new_tokens=8192,
    streamer=streamer,
)

🔧 Fine-Tuning

We recommend fine-tuning LFM2.5 for your specific use case to achieve the best results.

📊 Performance

Improvements over LFM2-8B-A1B

Thanks to reasoning, scaled-up pre-training, and large-scale RL, LFM2.5-8B-A1B improves over its predecessor across the board:

Knowledge and instruction following

Math and agentic workflows

CPU Inference

GPU Inference

LFM2.5-8B-A1B is the fastest model in its size class, reaching 18.5K output tokens per second at high concurrency, over 1.6B tokens per day on a single H100.

📬 Contact

• Got questions or want to connect? Join our Discord community.
• If you are interested in custom solutions with edge deployment, please contact our sales team.

Citation

@article{liquidAI20268BA1B,
  author  = {Liquid AI},
  title   = {LFM2.5-8B-A1B: Personal Assistant On Your Laptop},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {www.liquid.ai/blog/lfm2-5-8b-a1b},
}

@article{liquidai2025lfm2,
  title   = {LFM2 Technical Report},
  author  = {Liquid AI},
  journal = {arXiv preprint arXiv:2511.23404},
  year    = {2025}
}

@article{smith2026inplace,
  title   = {In-Place Tokenizer Expansion for Pre-trained LLMs},
  author  = {Smith, Jimmy T.H. and Dakhran, Tarek and Cabrera, Alberto and Lee, Simon S. and Pak, Paul and Tadimeti, Aditya and Seyde, Tim and Labonne, Maxime and Amini, Alexander and Lechner, Mathias},
  journal = {arXiv preprint arXiv:2607.15232},
  year    = {2026},
}