LFM2.5-2.6B-EXL3-4bpw

LFM2.5-2.6B — EXL3 4 bpw

An EXL3 quantization of LiquidAI/LFM2.5-2.6B, converted on Apple Metal and tested with MLXL3.

Approximately 1.76 GB to download. This is a weight-only quantization of the original instruction/reasoning model, without fine-tuning, QAT or other behavioral edits.

Run with MLXL3

In MLXL3 Desktop, search for 0xzknw/LFM2.5-2.6B-EXL3-4bpw in the model manager and download it. Alternatively, import the downloaded repository folder.

mlxl3 download 0xzknw/LFM2.5-2.6B-EXL3-4bpw --name lfm2.5-2.6b-4bpw
mlxl3 run lfm2.5-2.6b-4bpw

The tokenizer and chat template are included. This is EXL3, not GGUF or the standard MLX affine quantization format. A runtime must support both EXL3 and the LFM2 architecture; plain Transformers loading is not sufficient. Compatibility was verified with MLXL3, not every EXL3 runtime.

Conversion

• Source revision: 654f9463ce32b05d0429d76fe1f580b27d4c1ac0.
• Converter: PonyExl3 at 8e7fa6b1556f59fc669e25087903b279b9b0346f, with a local MLXL3 LFM adapter and explicit serialized codebook markers.
• Quantizer: Metal LDLQ; MCG codebook; K=4 on all 166 linear projections.
• Quantized linear parameters: 2,434,793,472.
• Calibration: 2,048 input rows per projection from PonyExl3's bundled WikiText-2 excerpt, captured as two 1,024-token sequences using the BF16 source.
• Sigma regularization: 0.025; Hessian shrinkage: 0; buffer rows: 128; feedback rows: 16.
• Tied embeddings, norms and convolution kernels are unquantized and exported in FP16.

4 bpw refers to the quantized linear weights, not the effective average over all model parameters. The total size also includes FP16 weights, scales, metadata and tokenizer files.

The original license is retained in LICENSE. See NOTICE for the modification notice. This is an independent conversion, not an official Liquid AI release.

Local validation — 2026-09-06

Perplexity increased by approximately 0.77% on this sample. WinoGrande used seed 20260901, whole-sentence mean log-probability scoring, and the same sampled examples for both models. These small local sanity checks are not an exhaustive quality evaluation or standard full benchmark scores, and the one-question difference does not establish a quality improvement.

All 166 EXL3 projections, remaining source tensors, codebook markers and the safetensors index were checked. Three deterministic CLI generation smoke tests passed: arithmetic in French, a JSON object with specified fields, and a short Python function.

On one Apple M5 Mac running on battery, those three short CLI generations measured approximately 61–63 decode tokens/s, 89–130 ms TTFT after loading, and 1.81–1.84 GB MLX peak memory. These are short smoke-test observations, not sustained throughput guarantees or total system memory figures. Context length, prompt, cache state, hardware and power conditions affect performance.

Only the 4 bpw variant is provided in this repository.