Downloads · 30 days
0
Gemma4-E2B-ternary
AsadIsmail/Gemma4-E2B-ternary
Gemma4-E2B-ternary is a machine learning model from AsadIsmail. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for ternary-quant. The card lists the license as apache-2.0.
Gemma 4 E2B (2.3B effective / 5.1B with embeddings) with both text and vision backbones quantized to ternary weights. 357 transformer layers quantized across language model and vision encoder; weights stored as 2-bit…
Access
Public
Updated Apr 6, 2026
Repo size
2.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.npz2.3 GB · 100%
At a glance
- Library
- ternary-quant
- License
- apache-2.0
- Access
- Public
- Created
- Apr 6, 2026
- Updated
- Apr 6, 2026
- SHA
- f9b47288
Base models
Model card
From the Hugging Face model README
Gemma4-E2B-ternary
Gemma 4 E2B (2.3B effective / 5.1B with embeddings) with both text and vision backbones quantized to ternary weights. 357 transformer layers quantized across language model and vision encoder; weights stored as 2-bit packed ternary codes with per-group float16 scales. Supports image+text inputs (Gemma 4 multimodal). Load with ternary-quant for inference on GPU, CPU (auto BF16), or Apple Silicon.
Compression: 1.8× (text + vision backbone) vs FP16 | Quality: coherent output ✓ (text + image verified)
Usage
pip install ternary-quant
from ternary_quant.inference import load_ternary_model
# GPU — fastest (dequantize once, use cuBLAS)
model, tokenizer = load_ternary_model(
"AsadIsmail/Gemma4-E2B-ternary", device="cuda", runtime_mode="cached"
)
# GPU — lowest VRAM (~2.3 GB (use runtime_mode='gemlite' or 'triton'))
model, tokenizer = load_ternary_model(
"AsadIsmail/Gemma4-E2B-ternary", device="cuda", runtime_mode="gemlite"
)
# CPU — auto BF16 on Intel AVX2+
model, tokenizer = load_ternary_model(
"AsadIsmail/Gemma4-E2B-ternary", device="cpu", runtime_mode="cached"
)
# Apple Silicon
model, tokenizer = load_ternary_model(
"AsadIsmail/Gemma4-E2B-ternary", device="mps", runtime_mode="metal"
)
# Generate
from ternary_quant.inference import generate_text
output = generate_text(model, tokenizer, "Ternary quantization works by", max_new_tokens=100)
print(output)
Performance
| Hardware | Speed | Mode |
|---|---|---|
| NVIDIA GPU | 15.6 tok/s (CUDA, runtime_mode='cached') | runtime_mode='cached' (dequantize to FP16 once) |
| Intel CPU | 6.3 tok/s (Intel CPU, BF16 auto-detected) | runtime_mode='cached' (auto BF16) |
| Apple Silicon | — | runtime_mode='metal' |
Quantization details
| Property | Value |
|---|---|
| Base model | google/gemma-4-E2B-it |
| Quantized components | text_backbone + vision_backbone |
| Scheme | tritplane3 (sum of ternary planes) |
| Weight values | {-1, 0, +1} packed 2-bit (4 values per byte) |
| Group size | 32 (per-group float16 scale + offset) |
| FP16 VRAM | 4.0 GB |
| Packed VRAM | ~2.3 GB (use runtime_mode='gemlite' or 'triton') |
| Compression | 1.8× (text + vision backbone) |
| Quality | coherent output ✓ (text + image verified) |
Quantized with ternary-quant — the only PTQ tool that quantizes VLMs and seq2seq models.
- Library
- ternary-quant
- License
- apache-2.0
- Created
- Apr 6, 2026
- Updated
- Apr 6, 2026