gemma-4-12B-it-MLX-4bit

Gemma 4 12B IT — MLX 4-bit

This repository is Sawfwair's reproducible Apple Silicon conversion of google/gemma-4-12B-it. It is a quantized derivative, not a new base model or fine-tune.

The checkpoint was converted with MLX-VLM because Gemma 4 12B uses the unified multimodal architecture (gemma4_unified). The package preserves the vision and audio paths, Google's pinned July 2026 processor/tokenizer metadata, the 262,144-token context configuration, and the canonical tool-capable chat template.

Conversion

The committed converter pins Python and all serialization dependencies, checks the exact upstream revision and bytes before conversion, restores verified upstream metadata after conversion, and writes complete machine-readable provenance to MERERUN_CONVERSION.json.

Conversion tool versions:

• mlx-vlm==0.6.5
• mlx==0.32.0
• mlx-lm==0.31.3
• mlx-audio==0.4.4
• transformers==5.14.1
• safetensors==0.8.0
• huggingface-hub==1.24.0

Integrity

These independently produced weight shards are byte-identical to the current mlx-community/gemma-4-12B-it-4bit shards. Sawfwair publishes the conversion and provenance so releases can be reproduced and audited from the Google checkpoint rather than depending on an opaque repackaging step.

Validation

The converted artifact passed Mere's two deterministic Gemma 4 continuation cases against a real checkpoint:

• one completed tool call followed by an assistant continuation: pass
• two completed tool calls followed by an assistant continuation: pass
• total: 2/2

This validates the local mere.run request adapter and canonical chat-template path. It is not a broad quality, safety, speed, or multimodal benchmark.

Usage

With MLX-VLM on Apple Silicon:

python -m mlx_vlm.generate \
  --model Sawfwair/gemma-4-12B-it-MLX-4bit \
  --prompt "Explain why the sky is blue in three sentences." \
  --max-tokens 160 \
  --temperature 0

To reproduce the validated mere.run acceptance test from the codex/gemma4-template-update source branch:

hf download Sawfwair/gemma-4-12B-it-MLX-4bit \
  --local-dir ./gemma-4-12B-it-MLX-4bit

swift run mere.run model benchmark tool-continuations \
  --model-root ./gemma-4-12B-it-MLX-4bit \
  --log-responses \
  --json

Scope and limitations

• This is a lossy 4-bit quantization; it is not numerically equivalent to the upstream BF16 checkpoint.
• The reported 2/2 result covers tool-result continuation behavior only.
• Hardware compatibility and performance depend on the installed MLX and MLX-VLM versions and the available unified memory.
• Consult the upstream model card for intended use, limitations, safety, evaluation, and multilingual/multimodal details.

License and attribution

The upstream model is published by Google DeepMind under Apache License 2.0. This repository redistributes a quantized derivative under the same license. See the Google Gemma 4 12B IT model card for the authoritative upstream attribution, license metadata, use guidance, and limitations.

The conversion tooling and native acceptance benchmark live in sawfwair/mere-run.