gemma-4-31b-it-oQ8e-mtp-drafter

gemma-4-31b-it-oQ8e-mtp-drafter

MLX conversion of Google's official MTP ("assistant") drafter checkpoint for gemma-4-31B-it, converted from google/gemma-4-31B-it-assistant. Kept at bf16, matching Google's original release precision — drafters are small enough that quantizing them further trades draft-acceptance accuracy for little practical size/speed benefit.

Usage

Pair with the main model, dynamicagency/gemma-4-31b-it-oQ8e-mtp. This is a separate loadable model, not a set of tensors merged into the main checkpoint — Gemma 4's MTP mechanism loads a small assistant model alongside the main one at inference time rather than baking draft-head tensors into the same checkpoint (the approach some other model families use).

In oMLX, discover this model directory, then on the main model's settings:

{
  "vlm_mtp_enabled": true,
  "vlm_mtp_draft_model": "<local id of this drafter model>"
}

• Model type: gemma4_assistant
• Format: MLX safetensors, bfloat16