Downloads · 30 days
415
19% of all-time downloads
gemma-4-12B-it-MXFP8
OsaurusAI/gemma-4-12B-it-MXFP8
gemma-4-12B-it-MXFP8 is a image-text-to-text model from OsaurusAI. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as gemma.
<p align="center" <a href="https://osaurus.ai"<img src="./osaurus-x-banner.png" alt="Osaurus AI"</a </p
All-time downloads
2.2K
Public
Parameters
12B
26.8 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors13.4 GB · 100%
Parameter types
How the weights are stored.
U3210.9B · 91%
Try a prompt
Base models
Model card
From the Hugging Face model README
<p align="center">
<a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a>
</p>
<h3 align="center">Gemma 4 12B-it - MXFP8 (MLX)</h3>
<p align="center">Apple Silicon MLX bundle for Osaurus and compatible vMLX runtimes.</p>
<p align="center">
<a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>
<a href="https://huggingface.co/OsaurusAI"><img src="https://img.shields.io/badge/HF-OsaurusAI-yellow?logo=huggingface" alt="OsaurusAI"></a>
<a href="https://github.com/jjang-ai/jangq"><img src="https://img.shields.io/badge/GitHub-JANG_Source-blue?logo=github" alt="JANG source"></a>
</p>
Important update (2026-06-03 4:06 PM PDT): These weights were rebuilt with the verified Gemma 4 12B fix. If you downloaded this repository before 2026-06-03 4:06 PM PDT, delete the local copy and re-download.
Model Details
| Property | Value |
|---|---|
| Base model | google/gemma-4-12B-it |
| Architecture | Gemma 4 unified dense 12B, text + image/audio/video-capable metadata |
| Format | MLX safetensors |
| Quantization | OCP MXFP8, group size 32; tied embedding and multimodal embedders fp16 passthrough |
| Tied token embedding | fp16 passthrough (embed_tokens.weight is not quantized) |
| Multimodal embedders | fp16 passthrough |
| Package size | 13.39 GB |
| Shards | 13 safetensors shards |
| Chat template | Gemma 4 tool-aware template, no default no-thinking thought-channel tail |
Runtime Notes
These rebuilt bundles preserve the tied token embedding in fp16 while keeping the main projection weights quantized. This fixes the bad prior artifact where embed_tokens.weight was packed and scaled like a normal linear weight.
The bundle includes generation_config.json, chat_template.jinja, tokenizer_config.json, and processor_config.json for Osaurus/vMLX loading.
Loading
Use Osaurus for local Apple Silicon chat and multimodal workflows, or load the bundle in a compatible MLX runtime:
from mlx_lm import load, generate
model, tokenizer = load("OsaurusAI/gemma-4-12B-it-MXFP8")
print(generate(model, tokenizer, "Hello", max_tokens=128))
Verification
Local release check for this rebuild:
| Check | Status |
|---|---|
embed_tokens.weight dtype | fp16 |
embed_tokens.scales / embed_tokens.biases | absent |
| Quantized attention projections | packed uint32 |
| README front matter | valid Hugging Face YAML first |
| Re-download notice | present after YAML |
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- gemma4_unified
- License
- gemma
- Languages
- en
- Created
- Jun 3, 2026
- Updated
- Jun 3, 2026