Downloads · 30 days
29
6% of all-time downloads
gemma-4-12B-it-qat-4bit-gs32
michaellin/gemma-4-12B-it-qat-4bit-gs32
gemma-4-12B-it-qat-4bit-gs32 is a any-to-any model from michaellin. Use it for the any-to-any task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as apache-2.0.
This model is quantized from google/gemma-4-12B-it-qat-q40-unquantized using the command python -m mlxvlm convert --hf-path google/gemma-4-12B-it-qat-q40-unquantized -q --q-group-size 32 --q-bits 4 --mlx-path "./gemma…
All-time downloads
488
Public
Parameters
12B
7.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors7.5 GB · 100%
Parameter types
How the weights are stored.
U3212B · 100%
Base models
Model card
From the Hugging Face model README
This model is quantized from google/gemma-4-12B-it-qat-q4_0-unquantized using the command python -m mlx_vlm convert --hf-path google/gemma-4-12B-it-qat-q4_0-unquantized -q --q-group-size 32 --q-bits 4 --mlx-path "./gemma-4-12B-it-qat-4bit-gs32". It uses the correct group size of 32, which corresponds to the 32 nibbles (uint4) of Q4_0.
Simply put, the default of mlx-lm and mlx-vlm is 64 group size, which does not match the QAT parameters that Google is using, since that targets llama.cpp, which uses 32 by default. This quantization fixes that at the cost of slightly inflated quant size (scale parameter storage size is effectively doubled).
- Task
- Any-to-Any
- Library
- mlx
- Type
- gemma4_unified
- License
- apache-2.0
- Created
- Jul 5, 2026
- Updated
- Jul 5, 2026