Downloads · 30 days
6
11% of all-time downloads
LFM2.5-VL-3B-ffn7168
konic-labs/LFM2.5-VL-3B-ffn7168
LFM2.5-VL-3B-ffn7168 is a machine learning model from konic-labs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
A surgically compressed derivative of LiquidAI/LFM2.5-VL-3B: FFN width pruned 10752 → 7168 (−33%) + LoRA distillation recovery, BF16.
All-time downloads
55
Public
Parameters
2.5B
4.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.9 GB · 100%
At a glance
- License
- other
- Model type
- lfm2_vl
- Access
- Public
- Created
- Aug 13, 2026
- Updated
- Aug 13, 2026
- SHA
- 25a82f8e
Model card
From the Hugging Face model README
LFM2.5-VL-3B-ffn7168
A surgically compressed derivative of LiquidAI/LFM2.5-VL-3B: FFN width pruned 10752 → 7168 (−33%) + LoRA distillation recovery, BF16.
| Size | vs original | |
|---|---|---|
| Original BF16 | 6.25 GB | — |
| This checkpoint | 4.92 GB | −21% |
What was done (3-step pipeline, all reproducible)
- Wanda channel pruning: FFN intermediate 10752 → 7168 (uniform across all 30 layers), joint swiglu importance (w1/w3 rows + w2 cols, one shared mask) from multimodal calibration (wikitext-2 + COCO), equal text/image weight.
- Distillation recovery: r32 LoRA on all FFN linears trained to match the base model's distribution (KL over the base's top-64 logits + small NLL anchor) on 512 rows disjoint from evaluation data (two low-LR passes).
- Evaluation: our statistical integrity harness vs the base.
Quality vs base (quick suite, our harness)
| Axis | Value |
|---|---|
| text PPL ratio (e^ΔNLL) | 0.85 (≤1.05 gate: PASS) |
| mm PPL ratio | 0.81 (≤1.10 gate: PASS) |
| text kld_t64 | 0.257 (base = 0) |
| mm kld_t64 | 0.260 |
| generation chrF (text/mm) | 0.48 / 0.50 |
| integrity index (record-only) | 41.7 / 100 |
Deeper cut than the 8192 sibling: slightly more distribution drift, still no word-prediction drift (PPL gates pass).
Load
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
"konic-labs/LFM2.5-VL-3B-ffn7168", dtype="bfloat16")
License & provenance
- Derivative of
LiquidAI/LFM2.5-VL-3B(LFM Open License v1.0 — license text retained; modifications per this card). - Pipeline and full reports: https://github.com/konic-labs/lfm25-vl-3b-lab
- Siblings: LFM2.5-VL-3B-ffn8192, LFM2.5-VL-3B-ffn8192-int4.
- Type
- lfm2_vl
- License
- other
- Created
- Aug 13, 2026
- Updated
- Aug 13, 2026