Downloads · 30 days
16
28% of all-time downloads
HarmenWessels/Ministral-3-3B-Instruct-int4-symg128-ov
Ministral-3-3B-Instruct-int4-symg128-ov is a text generation model from HarmenWessels. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
mrfakename/Ministral-3-3B-Instruct-2512-Llamafied-TextOnly converted to the OpenVINO™ IR format with weights compressed to INT4 by NNCF. 2.0 GB.
Downloads · 30 days
16
28% of all-time downloads
All-time downloads
58
Public
Repo size
2 GB
Likes
0
Public
Click a slice to open those files.
.bin2 GB · 99%
From the Hugging Face model README
mrfakename/Ministral-3-3B-Instruct-2512-Llamafied-TextOnly converted to the OpenVINO™ IR format with weights compressed to INT4 by NNCF. 2.0 GB.
The chain is not a direct Mistral conversion:
mistralai/Ministral-3-3B-Instruct-2512 (Apache-2.0, a mistral3 VLM)
→ community llamafication by mrfakename: vision encoder stripped, re-expressed as a plain
LlamaForCausalLM, and upcast (Mistral ships these quantized by default)
→ this repo: int4 weight compression for OpenVINO.
So it is a quantization of an upcast of a community re-architecture. It was chosen because
mistral3 is not registered in optimum-intel, making a direct VLM export impossible
(optimum-intel PR #1659 is still open). Mistral also ships no HF chat template, so the
chat template here is borrowed from unsloth's repo for the same base model; the tokenizer
encodes its control tokens ([INST], [THINK], [TOOL_CALLS]) as single tokens, which is what
makes the borrow vocab-compatible. The leaderboard numbers carry that caveat.
optimum-cli export openvino --model mrfakename/Ministral-3-3B-Instruct-2512-Llamafied-TextOnly --task text-generation-with-past --weight-format int4 --sym --group-size 128 --ratio 1.0 --awq --scale-estimation --dataset wikitext2 Ministral-3-3B-Instruct-int4-symg128-ov
Requires OpenVINO GenAI 2026.3.0.0 or newer. Symmetric int4 also keeps it eligible for the Intel NPU, which rejects asymmetric int4.
import openvino_genai as ov_genai
pipe = ov_genai.LLMPipeline("Ministral-3-3B-Instruct-int4-symg128-ov", "GPU")
cfg = ov_genai.GenerationConfig()
cfg.max_new_tokens = 512
cfg.do_sample = True
cfg.temperature = 0.1 # vendor-advised for this variant
cfg.top_p = 0.95
print(pipe.generate("Write a Python function that merges overlapping intervals.", cfg))
This model is measured in core-ultra-llm-server — per-task pass/fail (code generation, editing, agentic tool use, analysis, autocomplete) on an Intel Core Ultra 155H iGPU, re-run whenever the engine or server changes. The leaderboard there is authoritative; this card carries no numbers so it cannot go stale. Its benchmark card (decoding, think budget) is cards/HarmenWessels__Ministral-3-3B-Instruct-int4-symg128-ov.yaml.