LFM2.5-1.2B-Instruct-NVFP4

LFM2.5-1.2B-Instruct-NVFP4

NVFP4 (W4A4) quantisation of LiquidAI/LFM2.5-1.2B-Instruct, built and verified on NVIDIA GB10 Blackwell (sm_121a).

✅ The first NVFP4 build of any LFM2.5 checkpoint. Checked 2026-08-22: the only other FP4-family LFM2.5 build is mlx-community/LFM2.5-2.6B-mxfp4, which is MXFP4 for Apple MLX — a different format and a different runtime. Repository-content comparison only.

This is real W4A4, not weight-only

A weight-only NVFP4 export will not serve: SGLang's _is_fp4a4_nvfp4 returns False when input_quant is None, and there is no weight-only float4 scheme in the stack. Real W4A4 requires a calibrated input_global_scale per quantised Linear. This export has them:

format: nvfp4-pack-quantized      quant_method: compressed-tensors
weights:            4-bit float, tensor_group, group_size 16, static
input_activations:  4-bit float, tensor_group, group_size 16, dynamic="local"

The file

Ignore list — the LFM2 hybrid conv path is protected. Quantised: attention projections and SwiGLU MLPs. Left at bf16: all conv blocks (conv.in_proj / conv.conv / conv.out_proj, including 3-D kernels), every norm, and the tied embeddings. Quantising recurrent/conv state is the classic way to destroy this family.

Measured throughput

Sample output

/v1/completions, temperature 0, run twice — byte-identical both runs.

mathematical achievements is the Babylonian tablet known as the Yongkash tablet, which dates back to 2200 BC. The ancient Egyptians also made significant contributions to mathematics, particularly in the field of geometry. The Rhind Mathematical Papyrus, dating from around 1550 BC, is another important ancient mathematical text.

Requirements

Needs a runtime with compressed-tensors NVFP4 W4A4 support on Blackwell — SGLang v0.5.18-cu130 was used here; vLLM nightly also registers Lfm2ForCausalLM.

python3 -m sglang.launch_server --model-path <this-repo> \
  --host 0.0.0.0 --port 30000 --tp-size 1 --mem-fraction-static 0.45

Provenance

llmcompressor oneshot with QuantizationModifier(targets="Linear", scheme="NVFP4") — not model_free_ptq, which cannot calibrate activations and can only emit the unservable weight-only form. Calibration: HuggingFaceH4/ultrachat_200k (train_sft) via the model's own chat template. Source revision df58c174f05ff733f83f8cae10ea9298224c8006. Licence inherited from the base model.