LFM2.5-2.6B-Terminal-SFT

LFM2.5-2.6B Terminal SFT

Full-parameter supervised fine-tuning of LiquidAI/LFM2.5-2.6B for terminal-agent tasks.

The experiment reproduces the Phase 1 configuration in Liquid-CLI's train_unsloth_processed.py, changing only the base model and using native Transformers/TRL FlashAttention support for LFM2.5.

Training data

139,841 non-coding terminal-agent conversations prepared from the skill_based_easy, skill_based_medium, and skill_based_mixed configurations of nvidia/Nemotron-Terminal-Corpus using Liquid-CLI's prepare_data.py.

Training configuration

• Full-parameter BF16 fine-tuning
• Sequence length: 4,096
• Packing: enabled
• Assistant-only loss: enabled
• Per-device batch size: 4
• Gradient accumulation: 32 (effective batch size 128)
• Epochs: 1
• Learning rate: 2e-5
• Optimizer: AdamW 8-bit
• Warmup steps: 10
• Weight decay: 0.01
• Scheduler: linear
• Seed: 3407
• Attention: FlashAttention 2
• Hardware: 1x NVIDIA H100 80GB

Training results

• Optimizer steps: 1,092 / 1,092
• Final epoch: 1.0
• Training loss: 0.2867
• Mean token accuracy: 0.8947
• Tokens processed: 544,812,939
• Training runtime: 34,353 seconds (about 9 hours 33 minutes)

Source

• Experiment: https://github.com/gyunggyung/Liquid-CLI
• Base model: https://huggingface.co/LiquidAI/LFM2.5-2.6B
• Dataset: https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus