gemma-4-E2B-it

Gemma 4 E2B — Cactus engine bundle

Gemma 4 E2B instruction-tuned (vision and audio included), packaged for the Cactus on-device engine.

A small, on-device model is fast and private, but sometimes wrong. At Cactus we post-train models to know when they are wrong: we ship probes inside the checkpoint that score every answer with a confidence between 0 and 1, returned as structured data (never parsed out of the answer text). Answer on-device when confidence is high; re-route to a bigger model when it's low:

if confidence < 0.85:
    answer = ask_a_bigger_model(prompt)

Benchmarks

Gemma 4 E2B Hybrid, the smallest Gemma model, matches Gemini 3.1 Flash-Lite on most benchmarks by routing only 15–35% of queries to Flash-Lite and running the rest itself:

Quantisation quality is measured on Cactus Quants, which performs well at uniform quantization; developers are encouraged to benchmark Unsloth, GGUF, and MLX quantization independently.

Quickstart

# pip install cactus-compute
import json
from cactus.bindings.cactus import cactus_complete, cactus_init
from cactus.cli.download import download_bundle

lm = cactus_init(str(download_bundle("Cactus-Compute/gemma-4-E2B-it")))
result = cactus_complete(
    lm,
    [{"role": "user", "content": "What is the capital of France?"}],
    json.dumps({"max_tokens": 512, "auto_handoff": False}),
    None,
    lambda *_: None,
)
print(result["response"].strip())
print("confidence:", result["confidence"])

"auto_handoff": false returns the raw probe score so your app decides when to escalate; leave it on to let the engine hand off by itself.

Routing quality (AUROC)

AUROC measures how well the probe separates wrong answers from right ones (higher = better, 0.5 is random, 1.0 is perfect):

The strongest result: the probe was trained on zero audio data, yet achieves 0.79–0.88 AUROC on four audio benchmarks (two transcription, one audio MCQ, one out-of-domain transcription). This rules out surface-level explanations: the probe is reading a modality-independent correctness signal from the hidden state, not memorizing patterns from training data.

All formats

All Cactus Hybrid builds live in the Cactus Hybrid collection: Transformers · GGUF / llama.cpp · MLX · Cactus engine. Copy-paste quickstarts for every engine: github.com/cactus-compute/cactus-hybrid.

Gemma model use is subject to the Gemma terms.