gemma-4-12B-it-MLX-4bit

<center> <div style="display:flex; justify-content:center; align-items:center; gap:2%; max-width:560px; margin:0 auto;"> <a href="https://atomic.chat" style="flex:0 1 auto; min-width:0;"><img src="https://huggingface.co/AtomicChat/gemma-4-12b-it-MLX-4bit/resolve/main/pill_atomic_v3.png" alt="Atomic Chat" style="width:100%; height:auto; max-width:186px;"></a> <a href="https://discord.gg/8wGSsvmg4V" style="flex:0 1 auto; min-width:0;"><img src="https://huggingface.co/AtomicChat/gemma-4-12b-it-MLX-4bit/resolve/main/pill_discord_v3.png" alt="Join Discord" style="width:100%; height:auto; max-width:184px;"></a> <a href="https://github.com/AtomicBot-ai/Atomic-Chat" style="flex:0 1 auto; min-width:0;"><img src="https://huggingface.co/AtomicChat/gemma-4-12b-it-MLX-4bit/resolve/main/pill_github_v3.png" alt="GitHub" style="width:100%; height:auto; max-width:141px;"></a> </div> <br/> <img src="https://huggingface.co/AtomicChat/gemma-4-12b-it-MLX-4bit/resolve/main/hero.png" alt="Gemma 4 12B" style="width:100%; max-width:100%; height:auto; margin-bottom:0.6em;"/> <div style="display:flex; justify-content:center; gap:0.5em;"> <a href="https://huggingface.co/google/gemma-4-12B-it"><strong>Base model: google/gemma-4-12B-it</strong></a> </div> </center>

Gemma 4 12B, self-quantized to MLX by Atomic Chat. Built straight from Google's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline.

Highlights

• 11.95B parameters: the weights this repo quantizes.
• Context length: 256K tokens, as published by Google.
• 48 layers: Dense decoder, hybrid sliding-window (1024) and global attention.
• Modalities: Text, Image, Audio.
• Full imatrix ladder: every quant is calibrated with an importance matrix.
• Reasoning: All models in the family are designed as highly capable reasoners, with configurable thinking modes.
• Diverse & Efficient Architectures: Offers Dense and Mixture-of-Experts (MoE) variants of different sizes for scalable deployment.

[!NOTE] These MLXs are self-quantized from the original weights, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.

Model Overview

Benchmarks

Scores are Google's published results for the base google/gemma-4-12B-it, not our own measurements. Quantization preserves the large majority of this; Q4_K_M and up stay close to full precision.

Get started

• Atomic Chat: search AtomicChat/gemma-4-12b-it-MLX-4bit and hit Use this model.
• mlx-lm: mlx_lm.generate --model AtomicChat/gemma-4-12b-it-MLX-4bit --prompt "Hello" --max-tokens 512
• Server: mlx_lm.server --model AtomicChat/gemma-4-12b-it-MLX-4bit --port 8080

Best practices

Google's recommended sampling configuration for google/gemma-4-12B-it.

How these were made

1. Download google/gemma-4-12B-it (original weights).
2. Convert and quantize with mlx_lm.convert on our pipeline.

License

Original model by Google, released under the Apache 2.0 license. Full terms: Apache 2.0. Quantized by Atomic Chat.