Downloads · 30 days
0
LFM2.5-1.2B-Meditation
Nirav-Madhani/LFM2.5-1.2B-Meditation
LFM2.5-1.2B-Meditation is a text generation model from Nirav-Madhani. Use it when you need the model to write or continue text. The card lists the license as mit.
A fine-tuned version of LiquidAI/LFM2.5-1.2B-Thinking trained with a novel meditation phase: self-supervised mathematical introspection inserted between SFT and task RL.
Access
Public
Updated Apr 11, 2026
Repo size
17.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gz1.4 GB · 89%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Mar 24, 2026
- Updated
- Apr 11, 2026
- SHA
- 7bffd1ea
Try a prompt
Base models
Spaces
Model card
From the Hugging Face model README
LFM2.5-1.2B-Meditation
A fine-tuned version of LiquidAI/LFM2.5-1.2B-Thinking trained with a novel meditation phase: self-supervised mathematical introspection inserted between SFT and task RL.
Training Pipeline
What is Meditation?
Given a mathematical concept, the model produces a free-form exploration: restating concepts, probing edge cases, constructing examples/counterexamples, posing and solving novel problems, and synthesizing observations. This is scored by a composite reward:
Checkpoints
| File | Description |
|---|---|
| sft/sft.zip | SFT checkpoint (LoRA adapter, 42.4MB) |
| meditation_rl/checkpoint-60.tar.gz | Meditation RL step 60 (training in progress) |
Training Details
- Base model: LFM2.5-1.2B-Thinking (1.17B dense, hybrid conv+attention)
- Method: QLoRA (4-bit NF4, r=32, alpha=64, 22.2M trainable params)
- LoRA targets: q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3
- GRPO: K=8 generations, LR 5e-7, KL beta 0.04
- Judge: Gemini 3.1 Pro Preview (paid API)
- Hardware: Google Colab L4 (24GB VRAM, Ada Lovelace)
- Attention: PyTorch SDPA (built-in), bf16 compute
Current Metrics (Step 60)
| Metric | Value |
|---|---|
| Reward mean | 0.43 |
| KL divergence | 0.0009 |
| Step time | 77s |
| GPU memory | 3.7 / 22.5 GB |
Dataset
Training data: Nirav-Madhani/meditation-math-seeds
Paper
Working paper available in the repository.
- Task
- Text Generation
- License
- mit
- Created
- Mar 24, 2026
- Updated
- Apr 11, 2026