Downloads · 30 days
13
14% of all-time downloads
jarvis-pet/Qwen3.6-35B-A3B-int4-ov
Qwen3.6-35B-A3B-int4-ov is a image-text-to-text model from jarvis-pet. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for openvino. The card lists the license as apache-2.0.
An OpenVINO INT4 weight-only conversion of Qwen/Qwen3.6-35B-A3B — a vision-language MoE (hybrid linear-attention/SSM + full-attention layers, 256-expert mixture-of-experts, ~3 B active parameters per token). Produced…
Downloads · 30 days
13
14% of all-time downloads
All-time downloads
92
Public
Repo size
19.6 GB
Likes
0
Public
Click a slice to open those files.
.bin19.6 GB · 100%
From the Hugging Face model README
An OpenVINO INT4 weight-only conversion of
Qwen/Qwen3.6-35B-A3B — a vision-language MoE
(hybrid linear-attention/SSM + full-attention layers, 256-expert mixture-of-experts, ~3 B active
parameters per token). Produced with optimum-intel
openvino-genai.Created by L. Christensen
Released under the Apache License 2.0, inherited from the base model
Qwen/Qwen3.6-35B-A3B (whose LICENSE is Apache-2.0).
This is a derivative quantization only; all model rights and terms follow the base model.
Multi-component OpenVINO IR (~18.3 GB total), as emitted by the exporter:
| Component | Precision | Notes |
|---|---|---|
openvino_language_model.{xml,bin} | INT4 | the decoder (~17.4 GB) — the bulk |
openvino_text_embeddings_model.{xml,bin} | INT8 | |
openvino_vision_embeddings_model.{xml,bin} | INT8 | vision tower |
openvino_vision_embeddings_merger_model.{xml,bin} | INT8 | |
openvino_vision_embeddings_pos_model.{xml,bin} | INT8 | |
openvino_tokenizer.{xml,bin}, openvino_detokenizer.{xml,bin} | — | OpenVINO tokenizer IR |
config.json, openvino_config.json, generation_config.json, chat_template.jinja, tokenizer* | — |
Data-free, weight-only quantization — no calibration dataset, no GPTQ/AWQ, no scale estimation;
plain NNCF round-to-nearest weight compression as produced by optimum-cli ... --weight-format int4.
Exact per-module scheme (from openvino_config.json, dtype: "int4_int8_int8"):
group_size = 64, ratio = 1.0 (all eligible weights to
INT4), with an INT8-symmetric backup precision for the layers NNCF keeps at higher precision.# Toolchain (the transformers pin matters — see below)
pip install "transformers==5.2.0" "optimum-intel[openvino]" openvino-genai nncf
optimum-cli export openvino \
--model Qwen/Qwen3.6-35B-A3B \
--task image-text-to-text \
--weight-format int4 \
./qwen3.6-35b-a3b-int4-ov
Versions used for this conversion:
| Package | Version |
|---|---|
| OpenVINO | 2026.2.0 |
| openvino-genai | 2026.2.0.0 |
| NNCF | 3.2.0 |
| optimum / optimum-intel | 2.2.0 / 2.0.0 |
| transformers | 5.2.0 |
| Python | 3.14 |
transformers==5.2.0 is required for the export. Earlier releases don't recognize the
qwen3_5_moe architecture (AutoConfig raises "does not recognize this architecture"); some later
releases removed an internal symbol (Qwen3_5DynamicCache) that optimum-intel imports while building
the export patcher. 5.2.0 is the version that satisfies both. (Inference via openvino-genai does not
depend on this — the pin is only for the export.)--task image-text-to-text is required (it's a VLM). optimum-intel's OpenVINO exporter does not
register text-generation-with-past for qwen3_5_moe.Converting (not running) a 35 B model is RAM-heavy: the exporter loads the full bf16 weights and upcasts during tracing, so peak host memory is far larger than the ~18 GB INT4 output (on the order of ~200 GB for this model). On a machine with limited RAM, configure a large pagefile/swap for the conversion. Running the resulting INT4 IR needs far less (~30 GB resident on CPU).
import openvino_genai as ov_genai
pipe = ov_genai.VLMPipeline("qwen3.6-35b-a3b-int4-ov", "CPU") # or "GPU" / "AUTO"
cfg = ov_genai.GenerationConfig()
cfg.max_new_tokens = 2048
print(pipe.generate("Explain Rayleigh scattering in one sentence.", generation_config=cfg))
This is a reasoning model. The chat template opens a <think> block, so the generation is
…thinking…</think>…answer…. Split the output at </think> to separate the chain-of-thought
(reasoning_content) from the final answer (content). Keep thinking enabled (do not pass
enable_thinking=False) and allow a generous max_new_tokens — it commonly thinks 1000+ tokens
before answering.
Base model © the Qwen team — Qwen/Qwen3.6-35B-A3B.
Quantization tooling: OpenVINO, NNCF, and optimum-intel.