gemma-4-E4B-it-4bit

OsaurusAI/gemma-4-E4B-it-4bit

gemma-4-E4B-it-4bit is a any-to-any model from OsaurusAI. Use it for the any-to-any task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as gemma.

<p align="center" <a href="https://osaurus.ai"<img src="https://cdn-avatars.huggingface.co/v1/production/uploads/69d00705ce8872981c6c4fce/GWKjOwezSOhW5iuKpDwq.png" alt="Osaurus AI" width="120"</a </p

Downloads · 30 days

167

4% of all-time downloads

All-time downloads

4.1K

Public

Parameters

13.8 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors6.9 GB · 100%

Parameter types

How the weights are stored.

U327.5B · 94%

Base models

google/gemma-4-E4B-it

Model card

From the Hugging Face model README

<p align="center"> <a href="https://osaurus.ai"><img src="https://cdn-avatars.huggingface.co/v1/production/uploads/69d00705ce8872981c6c4fce/GWKjOwezSOhW5iuKpDwq_.png" alt="Osaurus AI" width="120"></a> </p> <h3 align="center">Gemma 4 E4B-it — 4-bit (MLX)</h3> <p align="center">Properly converted with all vision and audio tower weights verified intact</p> <p align="center"> <a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>  <a href="https://huggingface.co/OsaurusAI"><img src="https://img.shields.io/badge/HF-OsaurusAI-yellow?logo=huggingface" alt="OsaurusAI"></a> </p>

Why this exists: The mlx-community 8-bit conversion of Gemma 4 E4B has broken/zeroed-out vision tower weights, producing a model that appears functional for text but silently fails on image and audio inputs. This is a clean conversion from the original google/gemma-4-E4B-it with every multimodal weight tensor verified non-zero.

Model Details

Property	Value
Base Model	`google/gemma-4-E4B-it`
Parameters	4.5B effective (8B total with Per-Layer Embeddings)
Quantization	4-bit affine, mixed-precision (MLP layers kept at 8-bit)
Avg Bits/Weight	6.900
Model Size	6.4 GB
Architecture	Gemma 4 (text + vision + audio)
Context Length	128K tokens
Vocabulary	262K tokens

Multimodal Weight Verification

Every tensor in every multimodal component was loaded and checked for max(abs(tensor)) > 0. Zero broken weights found.

Component	Tensor Count	Status
Vision Tower (SigLIP)	658	All non-zero
Audio Tower (Conformer)	751	All non-zero
Language Model	1,485	All non-zero
Total	2,894	All verified

Mixed-Precision Quantization

mlx-vlm's default quantization predicate automatically keeps MLP gate/up/down projections at 8-bit across all 42 language model layers while quantizing attention and other weights to 4-bit. This improves quality over naive uniform 4-bit quantization.

Usage

# Requires Osaurus (https://osaurus.ai)
osaurus serve OsaurusAI/gemma-4-E4B-it-4bit

# Python API
from mlx_vlm import load, generate

model, processor = load("OsaurusAI/gemma-4-E4B-it-4bit")

# Text-only
output = generate(model, processor, "Explain quantum computing", max_tokens=500)

# With image
output = generate(model, processor, "Describe this image", ["path/to/image.jpg"], max_tokens=500)

Conversion Details

Detail	Value
Tool	`mlx-vlm` v0.4.4
Source dtype	bfloat16
Quantization mode	affine
Group size	64
Source	`google/gemma-4-E4B-it` (original Google release)

<p align="center">Converted by <a href="https://osaurus.ai">Osaurus AI</a></p>

Task: Any-to-Any
Library: mlx
Type: gemma4
License: gemma
Languages: en
Created: Apr 6, 2026
Updated: Aug 31, 2026