gemma-4-12B-it-qat-oQ4e-mtp

gemma-4-12B-it-qat-oQ4e-mtp

This is an oQ4e MLX conversion produced with oMLX.

Source and credit

• Original local source checkpoint: gemma-4-12B-it-qat-q4_0-unquantized
• Source repository / credit: https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-unquantized

Please follow the original model's license, acceptable-use policy, and attribution requirements.

Conversion

• Quantization: oQ4e (enhanced=True)
• Importance calibration: imatrix, 128 samples × 512 tokens
• oMLX version: 0.5.7

oQe uses an imatrix sensitivity calibration to assign precision selectively, rather than applying a uniform bit-width to every tensor. The generated oq_imatrix_report.json records the resulting quantization allocation.

python run_oqe_conversion.py gemma-4-12B-it-qat-q4_0-unquantized --models-dir /path/to/models --oq-level 4.0 --imatrix-samples 128 --imatrix-seq-length 512 --mtp-assistant /path/to/gemma-4-12B-it-qat-q4_0-unquantized-assistant --source-repo https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-unquantized

Lightning MTP

The Gemma 4 assistant checkpoint was merged as an oMLX Lightning MTP head. The assistant was kept at its shipped precision and was not independently quantized.

• Assistant checkpoint: gemma-4-12B-it-qat-q4_0-unquantized-assistant

Files

• model*.safetensors: quantized MLX weights
• model.safetensors.index.json: shard index
• config.json: model configuration
• oq_imatrix_report.json: imatrix calibration and quantization report