Qwen3-4B-Computer-Science-OpenVINO-INT4

Qwen3-4B-Computer-Science OpenVINO INT4

This repository provides an OpenVINO INT4 version of Qwen3-4B-Computer-Science, optimized for efficient inference on Intel CPUs and other OpenVINO-supported hardware.

The model has been exported using Optimum Intel with OpenVINO IR format and INT4 weight compression, providing a significantly smaller footprint while maintaining strong performance for software engineering and computer science workloads.

---

Model Information

---

Features

• OpenVINO IR optimized for CPU inference
• INT4 asymmetric weight compression
• Reduced storage requirements
• Lower memory usage
• Fast local inference
• Compatible with Hugging Face Transformers through Optimum Intel
• Includes tokenizer, detokenizer, and chat template

---

Export Configuration

The model was exported using:

optimum-cli export openvino \
    --model Irfanuruchi/Qwen3-4B-Computer-Science \
    --task text-generation-with-past \
    --weight-format int4 \
    Qwen3-4B-Computer-Science-OpenVINO-INT4

Compression statistics:

• 252 transformer layers compressed to INT4 asymmetric (group size 128)
• 1 auxiliary layer stored as INT8 per-channel

---

Installation

pip install -U openvino optimum-intel transformers

---

Example Usage

from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM

model_id = "Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-INT4"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = OVModelForCausalLM.from_pretrained(
    model_id,
    device="CPU",
)

messages = [
    {
        "role": "system",
        "content": "You are a computer science assistant."
    },
    {
        "role": "user",
        "content": "Explain Floyd's cycle detection algorithm."
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(response)

---

Validation

The exported model has been successfully validated using:

• OpenVINO 2026.2.1
• CPU inference
• Chat template support
• Greedy decoding
• Software engineering benchmark prompts

Validation confirmed successful generation of technically correct programming responses, including algorithm implementation and complexity analysis.

---

Intended Use

This model is intended for:

• Software engineering assistance
• Computer science education
• Code generation
• Code review
• Debugging
• Algorithm design
• Technical documentation
• Programming question answering

---

Limitations

As with other large language models, outputs should be reviewed before production use. The model may occasionally:

• Generate incorrect or incomplete code
• Hallucinate APIs or implementation details
• Produce inefficient solutions
• Misinterpret ambiguous instructions

INT4 compression may also introduce minor differences compared to higher-precision variants.

---

License

This model is distributed under the Apache License 2.0.

Please refer to the included LICENSE file for the complete license text and attribution requirements.