LFM2.5-2.6B-NVFP4

LFM2.5-2.6B-NVFP4

NVFP4 (W4A4) quantisation of LiquidAI/LFM2.5-2.6B, built and verified on NVIDIA GB10 Blackwell (sm_121a).

✅ The first NVFP4 build of any LFM2.5 checkpoint. Checked 2026-08-22: the only other FP4-family LFM2.5 build is mlx-community/LFM2.5-2.6B-mxfp4, which is MXFP4 for Apple MLX — a different format and a different runtime. Repository-content comparison only.

This is real W4A4, not weight-only

A weight-only NVFP4 export will not serve: SGLang's _is_fp4a4_nvfp4 returns False when input_quant is None, and there is no weight-only float4 scheme in the stack. Real W4A4 requires a calibrated input_global_scale per quantised Linear. This export has them:

format: nvfp4-pack-quantized      quant_method: compressed-tensors
weights:            4-bit float, tensor_group, group_size 16, static
input_activations:  4-bit float, tensor_group, group_size 16, dynamic="local"

The file

Ignore list — the LFM2 hybrid conv path is protected. Quantised: attention projections and SwiGLU MLPs. Left at bf16: all conv blocks (conv.in_proj / conv.conv / conv.out_proj, including 3-D kernels), every norm, and the tied embeddings. Quantising recurrent/conv state is the classic way to destroy this family.

Measured throughput

2.06× bf16, measured — not estimated.

Sample output

At the manufacturer-recommended sampling from Liquid's own generation_config.json (temperature 0.1, top_k 50, repetition_penalty 1.1). See the greedy note below.

the state of a quantum system. The wave function is not directly observable, but its probability density can be measured.

Let me provide an example with a simple harmonic oscillator: the time-dependent Schrödinger equation for a 1D harmonic oscillator…

⚠️ Use the recommended sampling, not bare greedy

At temperature=0 this base model falls into repetition loops. That is not a quantisation artifact: a bf16 control run on the identical prompt loops the same way, emitting meta-tokens like "(Repeat this sentence twice)". Liquid's own generation_config.json ships temperature 0.1, top_k 50, repetition_penalty 1.1, do_sample true — plain greedy is outside the recommended configuration. At those settings all five test prompts were coherent.

Requirements

Needs a runtime with compressed-tensors NVFP4 W4A4 support on Blackwell — SGLang v0.5.18-cu130 was used here; vLLM nightly also registers Lfm2ForCausalLM.

python3 -m sglang.launch_server --model-path <this-repo> \
  --host 0.0.0.0 --port 30000 --tp-size 1 --mem-fraction-static 0.45

Provenance

llmcompressor oneshot with QuantizationModifier(targets="Linear", scheme="NVFP4") — not model_free_ptq, which cannot calibrate activations and can only emit the unservable weight-only form. Calibration: HuggingFaceH4/ultrachat_200k (train_sft) via the model's own chat template. Source revision a334ee78cd38458bb71eda24109ac42dcec1309d. Licence inherited from the base model.