Downloads · 30 days
27
2% of all-time downloads
gemma-4-26B-A4B-it-4bit
OsaurusAI/gemma-4-26B-A4B-it-4bit
gemma-4-26B-A4B-it-4bit is a image-text-to-text model from OsaurusAI. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as gemma.
<p align="center" <a href="https://osaurus.ai"<img src="https://cdn-avatars.huggingface.co/v1/production/uploads/69d00705ce8872981c6c4fce/GWKjOwezSOhW5iuKpDwq.png" alt="Osaurus AI" width="120"</a </p
All-time downloads
1.1K
Public
Parameters
25.8B
15.6 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors15.6 GB · 100%
Parameter types
How the weights are stored.
U3225.2B · 98%
Try a prompt
Base models
Model card
From the Hugging Face model README
<p align="center">
<a href="https://osaurus.ai"><img src="https://cdn-avatars.huggingface.co/v1/production/uploads/69d00705ce8872981c6c4fce/GWKjOwezSOhW5iuKpDwq_.png" alt="Osaurus AI" width="120"></a>
</p>
<h3 align="center">Gemma 4 26B-A4B-it — 4-bit (MLX)</h3>
<p align="center">Mixed-precision 4-bit quantization with verified vision tower weights</p>
<p align="center">
<a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>
<a href="https://huggingface.co/OsaurusAI"><img src="https://img.shields.io/badge/HF-OsaurusAI-yellow?logo=huggingface" alt="OsaurusAI"></a>
</p>
Model Details
| Property | Value |
|---|---|
| Base Model | google/gemma-4-26B-A4B-it |
| Parameters | 26B total, 4B active (Mixture of Experts) |
| Quantization | 4-bit affine, mixed-precision (MLP layers kept at 8-bit) |
| Avg Bits/Weight | 4.843 |
| Model Size | 14.8 GB |
| Architecture | Gemma 4 (text + vision) |
| Context Length | 128K tokens |
| Vocabulary | 262K tokens |
Weight Verification
Every tensor in the vision tower was loaded and checked for max(abs(tensor)) > 0. Zero broken weights found.
| Component | Tensor Count | Status |
|---|---|---|
| Vision Tower (SigLIP) | 355 | All non-zero |
| Language Model (MoE) | 1,135 | All non-zero |
| Total | 1,490 | All verified |
Mixed-Precision Quantization
mlx-vlm's default quantization predicate automatically keeps MLP gate/up/down projections at 8-bit across all language model layers while quantizing attention and other weights to 4-bit. This improves quality over naive uniform 4-bit quantization.
Usage
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("OsaurusAI/gemma-4-26B-A4B-it-4bit")
# Text
prompt = apply_chat_template(processor, model.config, "Write a haiku about cats.")
output = generate(model, processor, prompt, max_tokens=200)
print(output.text)
# Vision
prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
output = generate(model, processor, prompt, image="photo.jpg", max_tokens=200)
print(output.text)
Conversion
Converted from google/gemma-4-26B-A4B-it using mlx-vlm v0.4.4:
mlx_vlm.convert --hf-path google/gemma-4-26B-A4B-it \
--mlx-path gemma-4-26b-a4b-it-4bit \
-q --q-bits 4 --q-group-size 64 --q-mode affine --dtype bfloat16
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- gemma4
- License
- gemma
- Languages
- en
- Created
- Apr 7, 2026
- Updated
- Apr 7, 2026