Downloads · 30 days
10
14% of all-time downloads
stijnvdoornum/Qwen3-8B-int4-ov
Qwen3-8B-int4-ov is a text generation model from stijnvdoornum. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Qwen/Qwen3-8B converted to OpenVINO IR with int4 weight compression via optimum-intel / NNCF.
Downloads · 30 days
10
14% of all-time downloads
All-time downloads
71
Public
Repo size
4.9 GB
Likes
0
Public
Click a slice to open those files.
.bin4.9 GB · 100%
From the Hugging Face model README
Qwen/Qwen3-8B converted to OpenVINO IR with int4 weight compression via optimum-intel / NNCF.
from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer
model_id = "stijnvdoornum/Qwen3-8B-int4-ov"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = OVModelForCausalLM.from_pretrained(model_id) # device="GPU" on Intel iGPU/NPU hosts
messages = [{"role": "user", "content": "What is OpenVINO?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
optimum-cli export openvino --model Qwen/Qwen3-8B \
--task text-generation-with-past --weight-format int4 \
--group-size 128 --ratio 1.0 <output_dir>
Converted with a small toolkit that also supports int8/fp16 export, AWQ/scale-estimation data-aware int4, and re-quantization of existing IRs.