LFM2.5-1.2B-Instruction-following

LFM2.5-1.2B-Instruction-following

Model Description / TL;DR

LFM2.5-1.2B-Instruction-following is a fine-tuned variant of Liquid AI's efficient LFM2.5 architecture, optimized for robust instruction following in noisy environments.

It is specifically engineered to parse through extreme human noise—typos, SMS shorthand, internet slang, and emojis—while maintaining the base model's lightning-fast inference speed and structured reasoning capabilities.

Note: This is a specialized tool for deterministic tasks (RAG, tool calling, JSON extraction) and was fine-tuned using LoRA on a small, curated dataset (800 samples) for demonstration and edge-deployment purposes. It is not a general-purpose SOTA model, nor is it a fine-tune of LLaMA.

Key Strengths

• Exceptional Noise Tolerance: Minimal performance drop on prompts with typos (teh), slang (u, pls, fr), and emojis (💀).
• Parsing Accuracy: Achieves ~10/10 accuracy in cleaning and understanding malformed inputs.
• Instruction Following: Scores approximately 7.0/10 on complex instruction adherence (measured on our internal stress-test suite).
• Efficient Adaptation: Uses Unsloth LoRA to update only ~18.28M parameters (1.5% of the base model).
• Deployment Ready: Ideal for edge devices and RAG pipelines due to sub-1GB memory footprint.

Training Methodology

The Supervised Fine-Tuning (SFT) run used the following exact configuration:

The LoRA adapter acts as an input "filter"—cleaning up the noise so the frozen reasoning engine can function deterministically.

Recommended Sampling Parameters

To achieve the best balance of accuracy and stability (avoiding hallucinations), use these strict, production-tested settings:

Why this works: The combination of top_k=20 and min_p=0.1 acts as a hard "stupidity firewall" that kills off extremely rare or off-topic tokens, ensuring the model picks the most statistically probable and correct token path. This is effectively greedy decoding with a tiny safety valve, perfect for RAG and tool-calling tasks.

For general chat, you can raise the temperature slightly (e.g., 0.4), but note that higher temperatures may increase the risk of hallucinations.

Usage Example (Python)

Load the model directly with the standard Hugging Face Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ertghiu256/LFM2.5-1.2B-Instruction-following"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

# Example: Parsing extreme noise
prompt = "teh capita l of france? pls respond quick"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    temperature=0.2,
    top_p=0.1,
    top_k=20,
    min_p=0.1
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Known Limitations

• Not a Knowledge Base: For factual recall (e.g., "Who won the 1999 World Cup?"), it relies on the base model's pretraining, which is modest. Use RAG for knowledge-heavy tasks.
• Creative Writing: This model is optimized for deterministic precision. At high temperatures (>0.6), it may produce creative but factually loose outputs.
• Complex Math: While it handles standard arithmetic well, complex multi-step math word problems may still be fragile if input syntax is heavily deformed.

Citation / Acknowledgments

• Liquid AI for releasing the incredibly efficient LFM2.5 architecture: Liquid AI Official
• Unsloth for making fine-tuning fast and memory-efficient: Unsloth AI
• Hugging Face for the transformers library and model card guidelines.

---

Built in 22 minutes. Runs in milliseconds. Deploy it anywhere. 🚀

• Developed by: ertghiu256
• License: apache-2.0
• Finetuned from model : unsloth/LFM2.5-1.2B-Instruct

This lfm2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

@article{liquidai2025lfm2,
  title={LFM2 Technical Report},
  author={Liquid AI},
  journal={arXiv preprint arXiv:2511.23404},
  year={2025}
}