lfm2.5-230m-coreML-fp16

LFM2.5-230M - Core ML (fp16)

A Core ML-optimized, 16-bit floating-point conversion of LiquidAI's LFM2.5-230M, designed for fast, on-device inference on Apple devices (iPhone, iPad, Mac). Retains full precision for maximum accuracy while enabling seamless integration with iOS/macOS apps via Core ML.

---

🔍 Model Overview

---

🎯 Capabilities

✅ On-Device Inference – Optimized for Apple Silicon (iPhone, iPad, Mac) via Core ML. ✅ Blazing Fast – 213 tok/s on Galaxy S25 Ultra, 42 tok/s on Raspberry Pi 5 (LiquidAI benchmarks). ✅ Efficient Architecture – Hybrid convolution + attention layers for speed and accuracy. ✅ Long Context – 128K token context window (32K extension phase included in training). ✅ General-Purpose – Text-only model trained on 19T tokens with distillation, DPO, and RL. ✅ Agentic-Ready – Designed for agent workflows, automation, and edge deployment.

---

📊 Performance Highlights

💡 Why LFM2.5?

• Faster than SSM hybrids & Gated Delta Networks of similar size.
• Runs everywhere: Cloud GPUs → CPUs → Mobile devices.
• Day-one ecosystem support: llama.cpp, MLX, vLLM, SGLang, ONNX, Core ML.

---

🚀 Quick Start

---

🍎 1. Use in iOS/macOS Apps (Core ML)

Prerequisites

• Xcode 15+
• macOS 14+ / iOS 17+
• Core ML framework

Step 1: Download the Model

git lfs install
git clone https://huggingface.co/code-and-canvas/lfm2.5-230m-coreML-fp16
cd lfm2.5-230m-coreML-fp16

Step 2: Integrate into Xcode

1. Drag model.mlpackage into your Xcode project.
2. Ensure it's added to your target's "Build Phases" → "Copy Bundle Resources".
3. Import Core ML in your Swift code:

import CoreML

// Load the model
guard let modelURL = Bundle.main.url(forResource: "model", withExtension: "mlpackage") else {
    fatalError("Model file not found")
}

do {
    let model = try MLModel(contentsOf: modelURL)
    // Use the model for inference
    let input = try MLDictionaryFeatureProvider(dictionary: ["input": "Your prompt here"])
    let prediction = try model.prediction(from: input)
    // Handle output
} catch {
    print("Error loading model: \(error)")
}

Step 3: Use with coremltools (Python)

import coremltools as ct

# Load the model
model = ct.models.MLModel("model.mlpackage")

# Run inference
input_data = {"input": "What is the capital of France?"}
prediction = model.predict(input_data)
print(prediction)

---

🐍 2. Use with Transformers (Hugging Face Format)

The repository includes both Core ML and Hugging Face formats (hf_model/).

Install Dependencies

pip install transformers torch

Load and Run

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "code-and-canvas/lfm2.5-230m-coreML-fp16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto"
)

# Generate text
inputs = tokenizer("Write a haiku about coding:", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Streaming Generation

from transformers import TextStreamer

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
outputs = model.generate(**inputs, streamer=streamer, max_new_tokens=100)

---

3. Command Line (llama.cpp)

If you convert to GGUF:

# Example (requires GGUF conversion first)
llama-cli -m lfm2.5-230m-fp16.gguf -p "Explain quantum computing simply."

---

🛠️ Configuration

Model Architecture

Generation Config (Default)

{
  "bos_token_id": 1,
  "eos_token_id": 7,
  "pad_token_id": 0,
  "do_sample": true,
  "temperature": 0.1,
  "top_k": 50,
  "repetition_penalty": 1.05,
  "use_cache": true
}

Special Tokens

---

📁 Files

---

🔧 Use Cases

✅ Recommended

• On-Device AI Apps – Deploy on iPhone, iPad, or Mac with Core ML.
• Edge Deployment – Run on Raspberry Pi, Jetson, or low-power devices.
• Agentic Workflows – Lightweight model for automation, chatbots, or tools.
• Local Inference – Fast, private inference without cloud dependency.
• Prototyping – Quickly test ideas offline or on-device.

⚠️ Considerations

💡 Why choose this over larger models?

• Speed: Optimized for real-time on-device inference.
• Size: Fits on mobile devices with limited storage.
• Efficiency: Lower power consumption and memory usage.

---

🔄 Training & Architecture Details

Base Model

• Original: LiquidAI/LFM2.5-230M-Base
• Pre-training: 19T tokens, including 32K context extension phase.

Post-Training (LFM2.5-230M)

1. Supervised Fine-Tuning (SFT) – Distilled from LFM2.5-350M.
2. Direct Preference Optimization (DPO) – Alignment for quality.
3. Multi-Domain Reinforcement Learning (RL) – Flexibility for downstream tasks.

Architecture (LFM2)

• Hybrid Design: Combines convolution (LIV blocks) and attention (GQA).
• Double-Gated LIV Convolution: 8 layers for efficient sequence processing.
• Grouped-Query Attention (GQA): 6 layers for scalable attention.
• Efficiency: Faster than SSM hybrids and Gated Delta Networks of similar size.

Core ML Conversion

• Precision: fp16 (16-bit floating point) for balance of speed and accuracy.
• Compatibility: Works on all Apple devices supporting Core ML.
• Format: .mlpackage (modern Core ML bundle format).

---

---

📜 License

Other – Refer to LiquidAI's terms for usage rights. (Original LFM2.5 models may have specific licensing; verify before commercial use.)

---

---

🙏 Acknowledgments

• Base Model: LiquidAI/LFM2.5-230M (Liquid AI).
• Architecture: LFM2 (Hybrid convolution + attention).
• Core ML: Apple Core ML framework.
• Conversion: Powered by coremltools and Hugging Face ecosystem.

---

---

💬 Example Prompts

General Q&A

What are the key differences between Python and JavaScript?

Coding

Write a Python function to reverse a linked list in-place.

Creative Writing

Write a short story about a robot discovering emotions.

Agentic Tasks

Act as a personal assistant. My calendar is empty tomorrow. Suggest 3 productive things I could do.

Long Context

Here's a 500-line Python script. Can you:
1. Summarize what it does.
2. Identify potential bugs.
3. Suggest improvements.
[Insert long script here...]

---

---

🐛 Issues & Support

• Bugs: Open an issue on the repository.
• Questions: Ask in Discussions.
• Contributions: PRs welcome!

---

---

📚 Additional Resources

• LiquidAI LFM2.5 Blog Post
• Core ML Documentation
• Hugging Face LFM2.5-230M
• MarkTechPost Coverage

---

🚀 Happy Building! Built with ❤️ by Code and Canvas.