Downloads · 30 days
13
20% of all-time downloads
jarvis-pet/gemma-4-26B-A4B-int4-ov
gemma-4-26B-A4B-int4-ov is a image-text-to-text model from jarvis-pet. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for openvino. The card lists the license as apache-2.0.
An OpenVINO INT4 weight-only (AWQ) conversion of google/gemma-4-26B-A4B-it — a vision-language mixture-of-experts model (~25 B total parameters, ~3.8 B active per token). Produced with optimum-intel + NNCF
Downloads · 30 days
13
20% of all-time downloads
All-time downloads
66
Public
Repo size
15.3 GB
Likes
1
Public
Click a slice to open those files.
.bin15.3 GB · 100%
From the Hugging Face model README
An OpenVINO INT4 weight-only (AWQ) conversion of
google/gemma-4-26B-A4B-it — a vision-language
mixture-of-experts model (~25 B total parameters, ~3.8 B active per token). Produced with
optimum-intel + NNCF
for efficient CPU (and Intel GPU) inference via
openvino-genai.
Released under the Apache License 2.0, inherited from the base model
google/gemma-4-26B-A4B-it (tagged apache-2.0).
This is a derivative quantization only; all model rights and terms follow the base model.
Multi-component OpenVINO IR (~14.3 GB total), as emitted by the exporter:
| Component | Precision | Notes |
|---|---|---|
openvino_language_model.{xml,bin} | INT4 (AWQ) | the decoder (~13.0 GB) — the bulk |
openvino_text_embeddings_model.{xml,bin} | INT8 | ~0.69 GB |
openvino_vision_embeddings_model.{xml,bin} | INT8 | vision tower (~0.53 GB) |
openvino_text_embeddings_per_layer_model.{xml,bin} | — | Gemma per-layer embedding scaling |
openvino_tokenizer.{xml,bin}, openvino_detokenizer.{xml,bin} | — | OpenVINO tokenizer IR |
config.json, openvino_config.json, generation_config.json, chat_template.jinja, tokenizer*, preprocessor_config.json | — |
Data-aware AWQ INT4 weight compression (NNCF, via optimum-intel). Exact per-module scheme
(from openvino_config.json, dtype: "int4"):
bits = 4, group_size = 64, ratio = 1.0, sym = false,
quant_method = "awq".ignored_scope.patterns: [".*router.*"], so the
expert-gating layers are not quantized to INT4 (preserves routing accuracy).The export requires a PRE-RELEASE toolchain. As of this writing the
gemma4OpenVINO exporter is not in any releasedoptimum-intel— it lives only on gitmain, and pairs with a specific transformers version + the nightly OpenVINO wheels. (Releasedoptimum-intelreturns "gemma4 is not supported yet"; transformers < 5.5 raisesKeyError: 'gemma4'.) The combination below is the one from the official OpenVINOgemma4notebook and is what produced this IR.
# Toolchain (pre-release — install into an isolated venv)
pip install "git+https://github.com/huggingface/optimum-intel.git" \
--extra-index-url https://download.pytorch.org/whl/cpu
pip install -U --pre openvino-genai openvino openvino-tokenizers nncf \
--extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install "transformers==5.5.0"
optimum-cli export openvino \
--model google/gemma-4-26B-A4B-it \
--task image-text-to-text \
--weight-format int4 \
./gemma-4-26B-A4B-int4-ov
Versions used for this conversion:
| Package | Version |
|---|---|
| OpenVINO | 2026.3.0.dev (nightly) |
| openvino-genai / openvino-tokenizers | 2026.3.0.dev (nightly) |
| NNCF | 3.2.0 |
| optimum / optimum-intel | 2.2.0 / git main |
| transformers | 5.5.0 |
| Python | 3.14 |
--task image-text-to-text is required — Gemma 4 is a VLM (model_type: gemma4,
Gemma4ForConditionalGeneration). The plain text task is not registered for this architecture.openvino-genai 2026.2 — you do not need
the nightly runtime to run it.import openvino_genai as ov_genai
pipe = ov_genai.VLMPipeline("gemma-4-26B-A4B-int4-ov", "CPU") # or "GPU" / "AUTO"
cfg = ov_genai.GenerationConfig()
cfg.max_new_tokens = 512
print(pipe.generate("Explain how a rocket engine works.", generation_config=cfg))
This is not a <think>-style reasoning model — it answers directly. When serving via a chat API,
supply Gemma 4's stop tokens / chat template explicitly; a bare VLMPipeline.generate may over-run the
intended end of turn without the model's EOS handling wired up.
Base model © Google — google/gemma-4-26B-A4B-it.
Quantization tooling: OpenVINO, NNCF, and optimum-intel.