gemma-4-E4B-it-ternary

Gemma 4-E4B-it — Ternary Quantized

Ternary-quantized version of google/gemma-4-E4B-it, produced with ternary-quant.

Gemma 4-E4B is Google's multimodal model that processes both images and text. This ternary-quantized version retains multimodal capabilities while reducing model size through a HuggingFace-native, component-aware PTQ workflow.

Quantization details

Usage

from ternary_quant.inference import load_ternary_model

# Load model (auto-detects best device: CUDA > MPS > CPU)
model, processor = load_ternary_model(
    "AsadIsmail/gemma-4-E4B-it-ternary",
    runtime_mode="metal"  # Use "cached" for NVIDIA GPU or CPU
)

# Image understanding
from PIL import Image
image = Image.open("photo.jpg")
inputs = processor(text="Describe this image", images=image, return_tensors="pt")
inputs = {k: v.to(model.device) for k, v in inputs.items()}
outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0], skip_special_tokens=True))

Hardware requirements

Reproduce

pip install ternary-quant
ternary-quant quantize-broad google/gemma-4-E4B-it \
    --output ./gemma-4-E4B-it-ternary \
    --components text_backbone multimodal_connector \
    --scheme tritplane3 \
    --dtype float16 \
    --eval

Part of the ternary-models collection

See github.com/Asad-Ismail/ternary-models for the full collection of ternary-quantized multimodal models.

Citation

@software{ternary_quant,
  author = {Ismail, Asad},
  title = {ternary-quant: Post-training ternary quantization for HuggingFace generative models},
  url = {https://github.com/Asad-Ismail/ternary-quant},
  year = {2026}
}