gemma-4-31B-it-r20260716-MLX-8bit

Gemma 4 31B-it — MLX 8-bit (2026-07-16 refreshed weights)

The first MLX quant of Google's July 16, 2026 stealth-refreshed Gemma 4 31B-it.

On 2026-07-15/16 Google re-pushed the Gemma 4 weights, chat templates, and kernels on Hugging Face without a version bump — fixing inconsistent tool-call JSON, syncing the canonical chat template, and sharpening vision OCR. Every public MLX conversion at the time of this upload was quantized from the pre-refresh weights (last modified ≤ 2026-07-06), so an MLX user re-pulling those repos still gets the old model.

This repo was quantized directly from google/gemma-4-31B-it after the refresh (upstream lastModified 2026-07-20).

Details

• Quantization: 8-bit via mlx_vlm.convert (mlx_vlm 0.6.0), 8.643 bits per weight, ~32 GB
• Modalities: text + vision (full vision tower included, standard MLX layout — loads with stock mlx_vlm.load, no sidecar files)
• Template: the refreshed canonical chat template, tool-definition rendering verified
• Verified on: Mac Studio M3 Ultra — 2 s load, text and vision generation checked

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("RockTalk/gemma-4-31B-it-r20260716-MLX-8bit")
config = load_config("RockTalk/gemma-4-31B-it-r20260716-MLX-8bit")

prompt = apply_chat_template(processor, config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["photo.jpg"], max_tokens=256))

Or serve it OpenAI-style:

python -m mlx_vlm.server --model RockTalk/gemma-4-31B-it-r20260716-MLX-8bit --port 8821

License

Gemma is provided under and subject to the Gemma Terms of Use. This repository redistributes a quantized derivative of google/gemma-4-31B-it; all upstream terms apply.

---

Quantized and verified on a Mac Studio M3 Ultra (512 GB) by RockTalk — a local-first fleet that believes when Google moves, Apple-Silicon users shouldn't have to wait.