LFM2.5-230M-SYNTH

LFM2.5-230M-Synth

Fine-tuned LiquidAI/LFM2.5-230M on a large synthetic reasoning dataset with preserved chain-of-thought (thinking) traces.

Model Details

Training Details

Training Results

Best eval loss: 1.6457 at step 22,000 (still improving at checkpoint time).

Loss decreased from 1.866 → 1.646 over 2.88B tokens — a 12% relative reduction with clear continued downward trend at the checkpoint boundary.

Chat Template

Uses the Liquid/LFM2 chat template with preserve_thinking=True. Reasoning traces from the dataset's reasoning_content field are mapped to the model's native thinking field before tokenization.

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mkurman/lfm25-230m-synth", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("mkurman/lfm25-230m-synth", trust_remote_code=True, dtype="bfloat16")

messages = [{"role": "user", "content": "Explain quantum entanglement simply."}]

# preserve_thinking=True so the model generates reasoning before its answer
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    preserve_thinking=True,
)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

This is a research model fine-tuned on synthetic reasoning data. It is intended for:

• Experimentation with small-model reasoning capabilities
• Chain-of-thought / thinking-trace generation
• Evaluation of synthetic data quality at small scale (230M)
• On-device or edge reasoning model prototyping

Limitations

• Partial training: This checkpoint is at ~0.86 epochs (step 22,000 / 36,500 planned). The full run continues.
• Small model: 230M parameters — not suitable for production deployment without further evaluation.
• Synthetic data only: Trained exclusively on synthetic reasoning traces; may exhibit distribution biases from the data generation pipeline.
• Limited context at training: Trained at seq=2048 despite the architecture supporting 128K. Long-context behavior is untested.

Checkpoint Info

This is checkpoint-22000 from a 36,500-step training run. The checkpoint includes:

• model.safetensors — 457 MB (weights only, no optimizer state)
• Full tokenizer files + chat template
• config.json with architecture details

Citation

If you use this model, please cite the base model:

@misc{liquid_lfm2,
  title={LFM2: Liquid Foundation Models},
  author={Liquid AI},
  year={2025},
  url={https://huggingface.co/LiquidAI/LFM2.5-230M}
}