Qwen3-4B-Computer-Science-Models

Qwen3-4B-Computer-Science OpenVINO INT4

This repository provides an OpenVINO INT4 deployment of Qwen3-4B-Computer-Science, optimized for efficient inference using Intel® OpenVINO™ and Optimum Intel.

The model is designed for software engineering and computer science workloads while significantly reducing storage requirements and memory usage through INT4 weight compression. It enables efficient deployment across OpenVINO-supported hardware while preserving the capabilities of the original model.

---

Model Information

* Supported execution devices depend on the installed OpenVINO Runtime, operating system, drivers, and available hardware. Depending on the platform, inference may be executed on supported CPUs, integrated GPUs, NPUs, or other OpenVINO-compatible accelerators.

---

Features

• OpenVINO IR deployment format
• INT4 asymmetric weight compression
• Reduced storage footprint
• Lower memory usage
• Efficient inference on OpenVINO-supported hardware
• Compatible with Optimum Intel and Hugging Face Transformers
• Exported tokenizer and detokenizer
• Chat template included
• Ready for local deployment

---

Export Configuration

The model was exported using:

optimum-cli export openvino \
    --model Irfanuruchi/Qwen3-4B-Computer-Science \
    --task text-generation-with-past \
    --weight-format int4 \
    Qwen3-4B-Computer-Science-OpenVINO-INT4

Compression summary:

• 252 transformer layers compressed using INT4 asymmetric with group size 128
• 1 auxiliary layer stored using INT8 per-channel

---

Installation

pip install -U openvino optimum-intel transformers

---

Example Usage

from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM

model_id = "Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-INT4"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = OVModelForCausalLM.from_pretrained(
    model_id,
    device="CPU",
)

messages = [
    {
        "role": "system",
        "content": "You are a computer science assistant."
    },
    {
        "role": "user",
        "content": "Explain Floyd's cycle detection algorithm."
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(response)

---

Validation

The exported model was successfully validated using:

• Intel® OpenVINO™ Runtime 2026.2.1
• Local OpenVINO inference
• Chat template support
• Greedy decoding
• Software engineering benchmark prompts

Validation confirmed successful generation of technically correct programming responses, including algorithm implementation and complexity analysis.

---

Intended Use

This model is intended for:

• Software engineering assistance
• Computer science education
• Programming support
• Code generation
• Code review
• Debugging
• Algorithm design
• Technical documentation
• Technical question answering

---

Model Family

The Qwen3-4B-Computer-Science release family currently includes:

Each release is maintained in its own repository with runtime-specific documentation, usage examples, integrity verification files, and configuration tailored to its target inference backend.

---

Credits

This release builds upon the work of several open-source projects and communities:

• Qwen Team for the Qwen3 foundation model architecture.
• Intel® OpenVINO™ Toolkit for the OpenVINO Runtime and deployment framework.
• Hugging Face for the Transformers ecosystem and the Optimum Intel integration.
• Optimum Intel for OpenVINO model export and runtime integration.

This repository provides an OpenVINO INT4 deployment of the original Qwen3-4B-Computer-Science model for efficient inference on OpenVINO-supported hardware.

---

Limitations

Like other large language models, this model may occasionally:

• Generate incorrect or incomplete code
• Hallucinate APIs or implementation details
• Produce inefficient implementations
• Misinterpret ambiguous instructions

INT4 weight compression may introduce minor quality differences compared to higher-precision variants.

All generated code should be reviewed and tested before use in production or safety-critical environments.

---

License

This repository is distributed under the Apache License 2.0.

This release is an OpenVINO INT4 conversion of the original Qwen3-4B-Computer-Science model and retains the licensing and attribution requirements applicable to the original project.

See the included LICENSE file for the complete license text.