Downloads · 30 days
25
26% of all-time downloads
HarmenWessels/K2-Horizon-3.7B-int4-symg128-ov
K2-Horizon-3.7B-int4-symg128-ov is a text generation model from HarmenWessels. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
IFM/K2-Horizon-3.7B — a dense reasoning model from the Institute of Foundation Models (released 2026-09) with graded thinking and native tool calling — converted to the OpenVINO™ IR format with weights compressed to I…
Downloads · 30 days
25
26% of all-time downloads
All-time downloads
98
Public
Repo size
3.3 GB
Likes
0
Public
Click a slice to open those files.
.bin3.2 GB · 99%
From the Hugging Face model README
IFM/K2-Horizon-3.7B — a dense reasoning model from the Institute of Foundation Models (released 2026-09) with graded thinking and native tool calling — converted to the OpenVINO™ IR format with weights compressed to INT4 by NNCF. 3.1 GB on disk.
The 3.7B ships layernorm_num_groups = 2 (grouped RMSNorm), which Llama cannot express, so the
custom modeling code is traced as-is. optimum-intel has no k2_horizon entry; the export
config (input and KV-cache signatures) is Llama's, so it is registered under the K2 model
type and the real CLI is driven in-process (loose kwargs to main_export are silently ignored):
from optimum.exporters.tasks import TasksManager
from optimum.exporters.openvino.model_configs import LlamaOpenVINOConfig
TasksManager.create_register("openvino", overwrite_existing=True)(
"k2_horizon", "text-generation", "text-generation-with-past")(LlamaOpenVINOConfig)
# then: optimum-cli export openvino --model IFM/K2-Horizon-3.7B --task text-generation-with-past
# --trust-remote-code --weight-format int4 --sym --group-size 128 --ratio 1.0
# --awq --scale-estimation --dataset wikitext2 K2-Horizon-3.7B-int4-symg128-ov
Environment: optimum-intel main (2.2.0.dev0+f2455b9), transformers 5.10.2 (K2's @strict
config needs >= 5.6), NNCF 3.3.0, OpenVINO 2026.3.1. configuration_k2_horizon.py is kept for
reference; OpenVINO GenAI does not need it.
Runs on OpenVINO GenAI 2026.3 or newer.
Two things about K2 that a plain LLMPipeline cannot do for you:
is sameas, HF {% generation %} masks).
If it sits in chat_template.jinja, GenAI refuses to construct the pipeline, so it ships
here as chat_template.vendor.jinja. Render it yourself with Jinja2 (strip the
generation tags, they are training-mask no-ops) and generate with
apply_chat_template=False.reasoning_effort picks the opener —
high -> <ifm|think>, medium -> <ifm|think_fast>, low -> <ifm|think_faster> — and the
closing tag mirrors it. The tags are added-but-not-special tokens, so the default decode keeps
them; split the answer on the closer.import re, jinja2, openvino_genai as ov_genai
d = "K2-Horizon-3.7B-int4-symg128-ov"
src = re.sub(r"\{%-?\s*(end)?generation\s*-?%\}", "",
open(f"{d}/chat_template.vendor.jinja", encoding="utf-8").read())
env = jinja2.Environment()
env.globals["raise_exception"] = lambda m: (_ for _ in ()).throw(ValueError(m))
prompt = env.from_string(src).render(
messages=[{"role": "user", "content": "Write a Python function that reverses a string."}],
add_generation_prompt=True, reasoning_effort="low")
pipe = ov_genai.LLMPipeline(d, "GPU")
cfg = ov_genai.GenerationConfig()
cfg.max_new_tokens = 1024
cfg.do_sample = True
cfg.temperature = 1.0 # vendor-recommended
cfg.top_p = 0.95
cfg.apply_chat_template = False
out = pipe.generate(prompt, generation_config=cfg)
reasoning, _, answer = out.partition("</ifm|think_faster>")
print(answer.strip())
Tool calls come back as
<ifm|tool_calls><ifm|tool_call>read_file\n<ifm|arg_key>path</ifm|arg_key>\n<ifm|arg_value>config.yaml</ifm|arg_value>\n</ifm|tool_call></ifm|tool_calls>
— string arguments unquoted, others as JSON, so type them by your tool's own schema. When you
feed tool history back, the template wants arguments as a dict (not OpenAI's JSON string) and
— on the 3.7B/7B — a thinking field (think / think_fast / think_faster /
reasoning_content, empty is fine) on every prior assistant turn, or it raises.
IFM recommends temperature 1.0, top_p 0.95 and at least 32,768 output tokens of reasoning. That last one is ~25 minutes per answer on an iGPU, so the leaderboard measures it at the lowest effort grade for non-reasoning tasks and an 8192-token think budget for analysis: K2 under a laptop budget, not K2 as intended.
The original upload's openvino_tokenizer.xml was built by openvino_tokenizers 2026.3, whose regex handling drops the \s+(?!\S) whitespace rule when the pre-tokenizer pattern contains \uXXXX escapes (K2's does): every Python indent tokenized as an N-space token plus a bare word instead of N−1 spaces + " word", on every prompt. The tokenizer IRs here are rebuilt with the escapes rewritten to PCRE2's \x{XXXX} and match the reference tokenizers library exactly. If you downloaded this model before 2026-09-18, re-download the two tokenizer files.
This model is measured in core-ultra-llm-server — per-task pass/fail (code generation, editing, agentic tool use, analysis, autocomplete) on an Intel Core Ultra 155H iGPU, re-run whenever the engine or server changes. The leaderboard there is authoritative; this card carries no numbers so it cannot go stale. Its benchmark card (decoding, think budget) is cards/HarmenWessels__K2-Horizon-3.7B-int4-symg128-ov.yaml.