LFM2.5_1.2B_Think_Enhanced

🧠 LFM2.5 1.2B Think Enhanced (Quantized)

A reasoning-enhanced and quantized version of the LFM2.5 1.2B model, optimized for structured thinking, mathematical reasoning, and efficient local inference.

---

🚀 Overview

• Base Model: LFM2.5 1.2B
• Fine-tuning: LoRA (QLoRA, 4-bit NF4)
• Focus: Chain-of-thought reasoning
• Quantization: 4-bit NF4 (BitsAndBytes)
• Context Length: 2048 tokens

---

🧪 Training Details

Method

• Supervised Fine-Tuning (SFT)
• QLoRA (4-bit NF4)
• Gradient checkpointing enabled

Hyperparameters

• Epochs: 50 (early stopping applied)
• Learning rate: 1e-4
• Batch size: 1
• Gradient accumulation: 64
• Optimizer: paged_adamw_8bit
• Scheduler: cosine
• Warmup steps: 100

---

📚 Training Data

This model is trained on high-quality reasoning datasets:

• TeichAI Claude Sonnet 4.6 Reasoning (1100 samples)
• TeichAI Claude Opus 4.6 Reasoning (887 samples)
• TeichAI Claude Opus 4.5 High Reasoning (250 samples)
• TeichAI Claude Sonnet 4.5 High Reasoning (250 samples)

Dataset Characteristics

• Step-by-step reasoning
• Structured thinking traces
• Problem solving oriented

---

🧠 Prompt Format

The model expects structured chat-style input:

---

💡 Reasoning Behavior

The model is trained to produce explicit reasoning steps.

Example sequence:

System: You are a helpful reasoning assistant.

User: Solve: What is 2 + 2?

Model:

<think> First, we identify... Then we compute... </think>

Final Answer: 4

---

⚙️ Usage

Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "bossman48/LFM2.5_1.2B_Think_Enhanced"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = """User:
Solve: What is the sum of first 100 natural numbers?

Assistant:
"""

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.3,
    top_p=0.8
)

print(tokenizer.decode(outputs[0]))