LFM2.5-230M-NVFP4

LFM2.5-230M NVFP4 Model Summary

The LFM2.5-230M NVFP4 model is a quantized version of Liquid AI's LFM2.5-230M, a compact hybrid language model designed for edge and on-device inference. This checkpoint was produced with NVIDIA Model Optimizer and exported in NVFP4 4-bit floating point format.

This release uses a conservative NVFP4 PTQ recipe: inner feed-forward linear operators are quantized, while attention, convolution blocks, embeddings, normalization layers, the language-model head, and the outer layers at both ends of the stack remain in higher precision. The scope was selected by measuring fidelity to the BF16 reference across a sweep of quantization scopes and keeping the fastest scope that stayed closest to the reference.

🚀 Key Specifications

• Architecture: Lfm2ForCausalLM, 14-layer hybrid architecture with LIV convolution and GQA blocks.
• Parameters: 230M.
• Base Model: LiquidAI/LFM2.5-230M.
• Precision: NVFP4 for selected feed-forward linear weights and activations.
• Quantization Scope: feed-forward linear operators in inner layers 5-11.
• Protected Modules: embeddings, lm_head, all self_attn* paths, all conv* paths, normalization layers, layers 0-4, and layers 12-13.
• Group Size: 16.
• Context Length: 32,768 tokens.
• KV Cache Quantization: none.
• License: inherits the base model's lfm1.0 license.
• Release Date: September 16, 2026.

🛠️ Software & Hardware Integration

• Quantization Tooling: NVIDIA Model Optimizer, official 0.46.0 release wheel.
• ModelOpt Version: 0.46.0.
• ModelOpt Commit: release tag 0.46.0, wheel nvidia_modelopt-0.46.0-py3-none-any.whl, sha256 1864b4e9921e287b065be3861ab48345144e673273ebb2b94bd9a6119a9eba8e.
• Torch: 2.10.0a0+b4e4ee81d3.nv25.12.
• Validated Runtime: vLLM 0.29.0 with --quantization modelopt_fp4.
• Validated FP4 Kernel: FlashInferCutlassNvFp4LinearKernel.
• Validated Hardware: NVIDIA GB10.
• Operating System: Linux.

Deployment Example, vLLM

pip install -U "transformers>=5.3.0" vllm

MODEL_ID="MarshallHD/LFM2.5-230M-NVFP4"

vllm serve "$MODEL_ID" \
    --quantization modelopt_fp4 \
    --trust-remote-code \
    --max-model-len 32768

Python Example, vLLM

from vllm import LLM, SamplingParams

model_id = "MarshallHD/LFM2.5-230M-NVFP4"

llm = LLM(
    model=model_id,
    quantization="modelopt_fp4",
    trust_remote_code=True,
    max_model_len=32768,
)

outputs = llm.generate(
    ["Explain NVFP4 quantization in one sentence."],
    SamplingParams(max_tokens=64, temperature=0.1),
)
print(outputs[0].outputs[0].text)

This checkpoint targets runtimes with ModelOpt FP4 support, in particular vLLM or SGLang with modelopt_fp4.

📊 Performance & Evaluation

Both evaluations below compare this checkpoint against the BF16 reference under identical engine settings: vLLM 0.29.0, greedy decoding, prefix caching disabled, BF16 KV cache, single NVIDIA GB10.

Fidelity to the BF16 reference

Teacher-forced scoring over 104,456 tokens drawn from three domains: general text, instruction prompts with reference answers, and Python function prompts with reference solutions.

Top-1 agreement is the share of scored positions where the most likely token matches the BF16 reference. KL divergence is measured on the top-32 support with the remaining mass pooled, which makes it a lower bound of the full-vocabulary value.

Task accuracy

Multiple-choice and short-answer accuracy over 24,063 examples, scored with EleutherAI lm-evaluation-harness 0.4.13 on the vLLM backend. Both columns use identical harness settings.

All values are accuracies reported as percentages. The weighted MCQA score is weighted by the number of examples in each benchmark.

Throughput

Fixed 128-token prompts and 128 generated tokens, median of three measured runs per setting.

📚 Datasets

Calibration Data

512 sequences of at most 512 tokens each, drawn from public English text and public instruction-following prompts with reference answers, rendered through the LFM2.5 chat template. The calibration set is disjoint from every evaluation set used above.

Evaluation Benchmarks

• WikiText-2: general text fidelity, test split.
• Instruction prompts with reference answers: chat-format fidelity.
• HumanEval prompts with reference solutions: code fidelity.
• MMLU-Pro: broad multiple-choice knowledge and reasoning.
• ARC-Challenge: science multiple-choice questions.
• HellaSwag: commonsense continuation selection.
• TruthfulQA MC1: truthfulness multiple-choice evaluation.

🔬 Quantization Details

The exported checkpoint includes hf_quant_config.json and config.json metadata for ModelOpt runtime detection.

{
  "quant_algo": "NVFP4",
  "group_size": 16,
  "kv_cache_quant_algo": null,
  "exclude_modules": [
    "lm_head",
    "model.embed_tokens",
    "model.layers.0*",
    "model.layers.1.*",
    "model.layers.10.self_attn*",
    "model.layers.11.conv*",
    "model.layers.12*",
    "model.layers.13*",
    "model.layers.2*",
    "model.layers.3*",
    "model.layers.4*",
    "model.layers.5.conv*",
    "model.layers.6.self_attn*",
    "model.layers.7.conv*",
    "model.layers.8.self_attn*",
    "model.layers.9.conv*"
  ]
}

Artifact Files

Citation

If you use this checkpoint, cite the base model and NVIDIA Model Optimizer:

@misc{liquidai_lfm25_230m,
  title = {LFM2.5-230M},
  author = {Liquid AI},
  year = {2026},
  url = {https://huggingface.co/LiquidAI/LFM2.5-230M}
}

@software{nvidia_modelopt,
  title = {NVIDIA Model Optimizer},
  author = {NVIDIA},
  url = {https://github.com/NVIDIA/Model-Optimizer}
}