LFM2-8B-A1B-int4-ov

LFM2-8B-A1B — OpenVINO INT4 (MoE, 8B total / 1B active)

INT4 group-quantized (gs128) OpenVINO export of LiquidAI/LFM2-8B-A1B, built with the OpenVINO 2026.3 toolchain (optimum-intel 2.1.0, transformers 5.4) — the export vintage that emits the tiled MoE expert layout ([experts, ofm, groups, group_size] constants).

Measured on an Intel Core Ultra 9 285K iGPU: loads in ~3.5 min, 27 tok/s decode at 4.5 GB weights.

MoE disk offload (OFFLOAD_RATIO) — read before assuming

OpenVINO 2026.3 can stream MoE expert weights from disk (LLMPipeline(dir, "GPU", OFFLOAD_RATIO=<0-100>)), shrinking resident GPU memory. This only works on XMX-capable GPUs (Arc dGPUs, Lunar Lake Arc 140V, Panther Lake — check that OPTIMIZATION_CAPABILITIES lists GPU_HW_MATMUL). On non-XMX iGPUs (desktop Arrow Lake, Meteor Lake) the property is a silent no-op — verified with this exact model: identical memory and speed at ratio 0 and 90 on a 285K.

Export command

optimum-cli export openvino --model LiquidAI/LFM2-8B-A1B \
  --weight-format int4 --group-size 128 --ratio 1.0 --trust-remote-code <out>

Usage

import openvino_genai as og
pipe = og.LLMPipeline("LFM2-8B-A1B-int4-ov", "GPU")  # add OFFLOAD_RATIO=50 on XMX GPUs
print(pipe.generate("Hello!", og.GenerationConfig(max_new_tokens=64)))

Or with NoLlama, which also ships scripts/offload-test.py for a ratio-0-vs-90 memory comparison on your GPU.

License

Inherits LiquidAI's LFM Open License v1.0 (see LICENSE): Apache-2.0-based, free use including commercial below US$10M annual revenue; above that, contact Liquid AI for a commercial license.