gemma-4-12B-it-ONNX

Gemma 4 12B (instruction-tuned) — ONNX for Transformers.js

A browser-ready ONNX export of google/gemma-4-12B-it, text only, for Transformers.js on WebGPU. Built for Oh my AI!, a chatbot that runs entirely in the browser.

As of September 2026, no other export of the 12B loads in Transformers.js: onnx-community publishes E2B and E4B only, and the existing 12B ONNX export targets onnxruntime-genai.

Files

Total download: about 8.2 GB. It stays under the ~15.75 GiB of ArrayBuffers a Chrome tab can hold, which Transformers.js needs to keep every file in memory while loading.

Usage

import { pipeline } from '@huggingface/transformers';

const generator = await pipeline('text-generation', 'Dramatik999/gemma-4-12B-it-ONNX', {
  device: 'webgpu',
  dtype: { embed_tokens: 'fp16', decoder_model_merged: 'q4f16' },
});
const out = await generator([{ role: 'user', content: 'Why is the sky blue?' }], { max_new_tokens: 128 });

How it was built

Exported from Google's safetensors with torch.onnx.export (dynamo, opset 18, transformers 5.17, torch 2.14), no custom operators. The script is public: export_transformersjs.py.

• The model declares gemma4_unified, unknown to Transformers.js 4.3. config.json presents it as gemma4 / Gemma4ForConditionalGeneration, which selects Transformers.js's text-only loading with two sessions. All computation lives in the graphs. The original type is kept under _source.
• The decoder only computes last-position logits, the only ones generation reads.
• The chat template is embedded in tokenizer_config.json, where Transformers.js's tokenizer reads it.

Validation

Against the PyTorch reference, greedy decoding:

Limitations

• Text only. The vision and audio encoders are not exported.
• Key/value cache. Sliding-window layers keep their whole past instead of cropping it to 1,024 tokens. Results are correct, but the cache grows by about 340 KB per token. Oh my AI! caps the context at 8,192 tokens.
• Speed. Attention runs as decomposed operators, without fused GroupQueryAttention.

License

Apache 2.0, like the original model.