Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT8

Note about the missing weights (2026-09-03)

For a long time this repository contained only this model card and none of the actual model files. That was my own mistake: back in June I created the repo and pushed the README, but I never actually completed the upload of the OpenVINO IR weights. I only noticed it now, while uploading other models.

The weights have now been uploaded. The export itself was never lost or corrupted — it had been sitting intact on my local machine the whole time.

Sorry to anyone who landed here earlier and found an empty repository.

Qwen3 4B Thinking 2507 Heretic CodeFeedback — OpenVINO INT8

This repository contains the OpenVINO INT8 export of:

JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback

The source model is a merged code-focused fine-tune based on:

JoaoZaokk/Qwen3-4B-Thinking-2507-MiniMax-M2.1-Distill-heretic

This version was converted from the full merged safetensors model to OpenVINO IR with INT8 weight compression.

Compared with the INT4 version, this INT8 export is expected to be larger and may use more VRAM, but it should preserve more of the original model quality.

Related repositories

Format

Conversion command

optimum-cli export openvino \
  --model ~/models-src/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback \
  --task text-generation-with-past \
  --weight-format int8 \
  --trust-remote-code \
  ./Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT8

The INT8 export does not need INT4-specific options such as --group-size 128 or --ratio 1.0.

Intended comparison

This INT8 version is mainly intended as a comparison build against:

JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT4

General expected tradeoff:

Actual results depend on OpenVINO version, Intel GPU driver, context length, prompt, batch size, and runtime configuration.

OpenArc load example

curl -X POST http://localhost:8000/openarc/load \
  -H "Content-Type: application/json" \
  -d '{
    "model_path": "/models/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT8",
    "model_name": "Qwen3-4B-Thinking-2507-Heretic-CodeFeedback-OpenVINO-INT8",
    "model_type": "llm",
    "engine": "ovgenai",
    "device": "GPU.0",
    "runtime_config": {}
  }'

Training background

The source model was trained with QLoRA/LoRA on Python and code instruction datasets, then merged back into the base model.

A SWE-smith trajectory experiment was tested separately, but it was not used in the final merged version.

LoRA configuration of source model

Target modules:

• q_proj
• k_proj
• v_proj
• o_proj
• gate_proj
• up_proj
• down_proj

Intended use

This INT8 OpenVINO version is intended for:

• OpenVINO quality testing
• comparison against INT4
• Intel Arc / Intel GPU inference
• Python code generation
• code explanation
• simple debugging
• instruction-following tests

Hardware notes

Important notes

This is not the original F16 model. This is an OpenVINO INT8 compressed export.

For further conversions, use the original merged safetensors repository as the master source:

JoaoZaokk/Qwen3-4B-Thinking-2507-Heretic-CodeFeedback

This is an experimental model. It may produce incorrect code, unsafe suggestions, or hallucinated explanations. Outputs should be reviewed before use in production or security-sensitive environments.