LFM2.5-1.2B-Instruct-int4-ov

LFM2.5-1.2B-Instruct - OpenVINO INT4

OpenVINO IR conversion of LiquidAI/LFM2.5-1.2B-Instruct (hybrid conv+attention, on-device class), quantized to INT4 asymmetric, group size 128 (data-free / NNCF defaults) for Intel Core Ultra iGPUs.

optimum-cli export openvino -m LiquidAI/LFM2.5-1.2B-Instruct \
  --task text-generation-with-past --weight-format int4 --group-size 128 --ratio 1.0 \
  LFM2.5-1.2B-Instruct-int4-ov

Usage (OpenVINO GenAI)

import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline("LFM2.5-1.2B-Instruct-int4-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Write a Python function that reverses a string.", max_new_tokens=128))

Fast, lightweight autocomplete/chat backend; tested end-to-end as an OpenAI-compatible Continue.dev backend via core-ultra-llm-server.

Provenance

• Base model: LiquidAI/LFM2.5-1.2B-Instruct (Liquid AI)
• Recipe: INT4 asymmetric, group_size 128, ratio 1.0, data-free (NNCF defaults)
• No finetuning - weights are a direct quantization of the original

Benchmarks

This model is measured in core-ultra-llm-server — per-task pass/fail (code generation, editing, agentic tool use, analysis, autocomplete) on an Intel Core Ultra 155H iGPU, re-run whenever the engine or server changes. The leaderboard there is authoritative; this card carries no numbers so it cannot go stale. Its benchmark card (decoding, think budget) is cards/HarmenWessels__LFM2.5-1.2B-Instruct-int4-ov.yaml.