Downloads · 30 days
26
9% of all-time downloads
LFM2.5-VL-3B-ffn8192-int4-MLX-optimized
konic-labs/LFM2.5-VL-3B-ffn8192-int4-MLX-optimized
LFM2.5-VL-3B-ffn8192-int4-MLX-optimized is a image-text-to-text model from konic-labs. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx-vlm. The card lists the license as apache-2.0.
5.1–5.6× faster decode on Apple Silicon (M3, 16 GB) with quality preserved.
All-time downloads
293
Public
Parameters
3.1B
1.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 99%
Parameter types
How the weights are stored.
U323.1B · 99%
Try a prompt
Base models
Model card
From the Hugging Face model README
LFM2.5-VL-3B-ffn8192-int4 — MLX Optimized
5.1–5.6× faster decode on Apple Silicon (M3, 16 GB) with quality preserved.
An inference-optimized MLX port of
konic-labs/LFM2.5-VL-3B-ffn8192-int4:
the compressed-tensors INT4/INT8 weights are mapped losslessly into MLX
QuantizedLinear layers (no dequantization, no re-quantization), fused at
load time, and driven by a lean custom decode loop.
⚠️ Fused checkpoint layout. The weights in this repo use fused names (e.g.
w1+w3,q+k+v) and require the loader in the companion code repo — not the stockmlx_vlm.utils.load.
Companion code
github.com/konic-labs/lfm25-mlx-optimized
— packer/loader (quant_map.py), lean decode loop (fastgen.py), REPL
(chat_lfm2.py), OpenAI-compatible server (serve_lfm2.py), benchmark and
quality gates.
Benchmarks
On Apple M3 (16 GB unified memory), greedy decode, 128 tokens, prompt
"Write a detailed paragraph about the history of the Pyrenees mountains."
(machine-load dependent; full run log in the code repo's .auto/log.jsonl).
| Metric | Source checkpoint (this repo's base) | This port | Δ |
|---|---|---|---|
| Decode throughput | 12.16 tok/s | 62–68 tok/s | +410–460% |
| Text TTFT | 167 ms | 62 ms | −63% |
| Image TTFT | ~1.25 s | ~0.88 s | −30% |
| Multi-turn TTFT (KV reuse) | ~188 ms | ~64 ms | −66% |
| Prefill | ~200 tok/s | ~600 tok/s | +200% |
| Checkpoint size | 2.79 GB | 1.63 GB | −42% |
| Peak RSS | ~4.8 GB | ~1.9 GB | −60% |
Optimizations
| Component | Bits | Notes |
|---|---|---|
| FFN w1/w3/w2 | INT4 | checkpoint-native; 4-bit is the quality floor — 3-bit deterministically breaks multi-turn tool calling |
| Conv in/out_proj | 4-bit | −185 MB decode bandwidth |
| Tied embeddings / lm_head | 4-bit | the fp16 128k×2048 matmul was the dominant decode cost |
| LM attention (q/k/v/o) | 4-bit | −33 MB decode bandwidth |
| Vision fc2 | INT8 padded | 4304→4352 zero-extended packed words, exact checkpoint values |
| Vision projector | 8-bit | |
patch_embedding | fp16 (kept) | 8-bit there destroys vision |
| Scales / biases | fp16 | checkpoint scales are natively fp16 |
Plus load-time linear fusion (w1+w3, q+k+v; bit-identical math,
−24% RSS), fp16 quantized_matmul outputs, and a lean decode loop:
no greedy logsumexp, argpartition+categorical sampling,
bool-sync EOS checks, last-position-only lm_head in prefill, and
prefix-verified prompt-cache KV reuse.
Usage
Requires Python ≥ 3.12, mlx, mlx-vlm, and the loader from the
companion code repo.
git clone [email protected]:konic-labs/lfm25-mlx-optimized.git
cd lfm25-mlx-optimized
.venv/bin/python chat_lfm2.py --model konic-labs/LFM2.5-VL-3B-ffn8192-int4-MLX-optimized
Or serve it as an OpenAI-compatible endpoint:
.venv/bin/python serve_lfm2.py --port 8080 --model konic-labs/LFM2.5-VL-3B-ffn8192-int4-MLX-optimized
Files
LFM2.5-VL-3B-ffn8192-int4-MLX-optimized/
config.json
model.safetensors
chat_template.jinja
tokenizer.json
tokenizer_config.json
generation_config.json
processor_config.json
README.md
Quality gates
The port passes automated quality gates: greedy text (Paris), greedy single-turn tool calling, image description (red square on white), and a 16-trial multi-turn tool-calling gate (discriminates the 3-bit cliff: ~40% call rate vs ~0% at 3-bit).
Research
The full experiment trail (15 kept runs, 12.16 → ~62–68 tok/s) is in the
code repo under .auto/ (playbook, run log, ideas, dead ends).
License
Apache-2.0 (model weights as per the base model).
- Task
- Image-Text-to-Text
- Library
- mlx-vlm
- Type
- lfm2_vl
- License
- apache-2.0
- Languages
- en
- Created
- Aug 15, 2026
- Updated
- Aug 15, 2026