Downloads · 30 days
29
16% of all-time downloads
HarmenWessels/granite-4.1-3b-int4-cw-ov
granite-4.1-3b-int4-cw-ov is a text generation model from HarmenWessels. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
OpenVINO IR conversion of ibm-granite/granite-4.1-3b (dense 40-layer transformer, 128k context, Apache-2.0), quantized to INT4 symmetric channel-wise with AWQ + scale-estimation calibration — the fast recipe for Intel…
Downloads · 30 days
29
16% of all-time downloads
All-time downloads
182
Public
Repo size
3.7 GB
Likes
0
Public
Click a slice to open those files.
.bin1.8 GB · 99%
From the Hugging Face model README
OpenVINO IR conversion of ibm-granite/granite-4.1-3b (dense 40-layer transformer, 128k context, Apache-2.0), quantized to INT4 symmetric channel-wise with AWQ + scale-estimation calibration — the fast recipe for Intel iGPUs, with data-aware calibration to preserve output quality.
Converted with optimum-cli (optimum-intel, transformers 4.57.6):
optimum-cli export openvino -m ibm-granite/granite-4.1-3b \
--task text-generation-with-past --weight-format int4 --sym --group-size -1 \
--awq --scale-estimation --dataset wikitext2 \
granite-4.1-3b-int4-cw-ov
An INT8 build is also available as the conservative choice.
import openvino_genai as ov_genai
pipe = ov_genai.LLMPipeline("granite-4.1-3b-int4-cw-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Write a Python function that merges overlapping intervals.",
max_new_tokens=256))
Tested end-to-end as an OpenAI-compatible Continue.dev backend via core-ultra-llm-server.
ibm-granite/granite-4.1-3b (IBM, Apache-2.0, released 2026-04)This model is measured in core-ultra-llm-server — per-task pass/fail (code generation, editing, agentic tool use, analysis, autocomplete) on an Intel Core Ultra 155H iGPU, re-run whenever the engine or server changes. The leaderboard there is authoritative; this card carries no numbers so it cannot go stale. Its benchmark card (decoding, think budget) is cards/HarmenWessels__granite-4.1-3b-int4-cw-ov.yaml.