LFM2.5-1.2B-Saiga-It-v2

LFM2.5-1.2B-Saiga-It-v2

My first LLM — an attempt to make a Russian-language Saiga out of LFM2.5 1.2B by Liquid AI.

Inspired by the amazing work of Ilya Gusev and the Saiga project.

What is this?

LFM2.5 1.2B is a compact but powerful model from Liquid AI with very dense knowledge packing. I tried to teach it Russian through CPT + SFT pipeline.

The result is... interesting.

Training

Stage 1 — Continued Pre-Training (CPT):

• wikimedia/wikipedia (Russian, ~350k articles)
• uonlp/CulturaX (Russian, 300k)
• allenai/c4 (Russian 200k + English 300k for retention)

Stage 2 — Supervised Fine-Tuning (SFT):

• IlyaGusev/saiga_scored (opus_score ≥ 8, ~27k examples)
• d0rj/alpaca-cleaned-ru (15k examples)
• IlyaGusev/ru_sharegpt_cleaned (243 examples, but what examples)
• IlyaGusev/ru_turbo_saiga
• lksy/ru_instruct_gpt4

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "NickupAI/LFM2.5-1.2B-Saiga-It-v2",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "NickupAI/LFM2.5-1.2B-Saiga-It-v2",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Привет! Как дела?"}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Honest Warning ⚠️

The model hallucinates badly. It may invent a planet called Gamma-Tit, introduce itself as Yandex or Irina, or confuse dwarf planets with kvass. This is not a bug — this is a feature.

Recommended for:

• Creative nonsense generation (lol)
• Experiments and research
• Inspiration and laughs

Not recommended for:

• Factual questions
• Medicine, law or any serious topics
• Astronomy (especially dwarf planets)

Example outputs

— Назови карликовые планеты солнечной системы

1. Марс, 2. Юпитер, 3. Сатурн, 4. Квас, 5. Гамма-тит...

Iconic.

What's next

Working on v3 — full Russian Wikipedia + Habr CPT, better SFT mix.