gemma-4-E2B-it-litert-lm

litert-community/gemma-4-E2B-it-litert-lm

Main Model Card: google/gemma-4-E2B-it

This model card provides the Gemma 4 E2B model in a way that is ready for deployment on Android, iOS, Desktop, IoT and Web.

Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. This particular Gemma 4 model is small so it is ideal for on-device use cases. By running this model on device, users can have private access to Generative AI technology without even requiring an internet connection.

These models are provided in the .litertlm format for use with the LiteRT-LM framework. LiteRT-LM is a specialized orchestration layer built directly on top of LiteRT, Google’s high-performance multi-platform runtime trusted by millions of Android and edge developers. LiteRT provides the foundational hardware acceleration via XNNPack for CPU and ML Drift for GPU. LiteRT-LM adds the specialized GenAI libraries and APIs, such as KV-cache management, prompt templating, and function calling. This integrated stack is the same technology powering the Google AI Edge Gallery showcase app.

LiteRT-LM uses a state of the art Gemma-4 mobile quantization scheme that uses a mixture of 2bit, 4bit and 8 bit weights. This means that for text only use cases the weight footprint in memory can be as low as 0.8 GB while the runtime uses memory mapping to support the 1.12GB of embedding parameters. This approach gives significant working memory savings on some platforms as seen in the more detailed data below. Additionally the Vision and Audio models are loaded on demand to further reduce memory consumption.

Try Gemma 4 E2B

<div align="center"> </div>

Build with Gemma 4 E2B and LiteRT-LM

Ready to integrate this into your product? Get started here.

Gemma 4 E2B Performance on LiteRT-LM

All benchmarks were taken using 1024 prefill tokens and 256 decode tokens with a context length of 2048 tokens via LiteRT-LM. The model can support up to 32k context length. The inference on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads. Time-to-first-token does not include load time. Benchmarks were run with caches enabled and initialized. During the first run, the latency and memory usage may differ. Model size is the size of the file on disk.

CPU memory was measured using, rusage::ru_maxrss on Android, Linux and Raspberry Pi, task_vm_info::phys_footprint on iOS and MacBook and process_memory_counters::PrivateUsage on Windows.

Android

Note: On supported Android devices, Gemma 4 is available through Android AI Core as Gemini Nano, which is the recommended path for production applications.

Android with Speculative Decoding

The numbers in this section include speculative decoding. Speculative decoding is an optimization that accelerates LLMs by using a small, fast "draft" model to quickly predict multiple upcoming tokens, while a larger “target” model then verifies those tokens in parallel. The effectiveness of speculative decoding is task dependent because the “draft” model can more easily predict the correct tokens of some tasks. The metrics in this section were collected from a variety of sample prompts and grouped into categories by task type. The baseline measurements are an average across all task types. The number of input and output tokens varied across prompts. Note that if you download this model before May 5, 2026, you should re-download the model if you want to use speculative decoding. Speculative decoding is available on CPU and GPU on Mobile and Desktop.

iOS

Linux

macOS

Windows

Web

<small>

• Web on LiteRT-LM uses a specially optimized model for Web because of its unique memory constraints. The model can support up to 128k context length.

</small>

IoT

<small>

• NPU model is benchmarked with 4096 context length

</small>

Running Gemma 4 E2B on Web with LiteRT-LM.js

See our LiteRT-LM.js overview to get started using the gemma-4-E2B-it-gpu.litertlm model file.

🔨 HACKATHON: Qualcomm QCS8275 NPU variant (Arduino Ventuno Q) 🔨

gemma-4-E2B-it_Qualcomm_QCS8275.litertlm is the ahead-of-time compiled NPU variant for Qualcomm QCS8275 (Hexagon v75), used by the Google × Qualcomm Arduino LiteRT hackathon's Ventuno Q example. It is a separate artifact from gemma-4-E2B-it.litertlm; use the NPU variant with the matching Qualcomm dispatch/runtime libraries, not with the CPU backend. Compatibility with other Qualcomm chips or Android runtimes is not established by this example.

Download and run

Download with the Hugging Face CLI into the hackathon checkout's models/ directory:

hf download litert-community/gemma-4-E2B-it-litert-lm \
  gemma-4-E2B-it_Qualcomm_QCS8275.litertlm \
  --local-dir models

Follow the example's board setup instructions before running inference. Its scripts/setup_npu.py retrieves the matching QAIRT libraries and a pinned Linux ARM64 LiteRT-LM native runtime/Qualcomm dispatch bundle from native-v0.18.0-e. The example uses that native runtime in place of the Python wheel's bundled library; installing the Python package alone is not the complete setup. Qualcomm SDK libraries are downloaded separately under their own terms and are not included in this model file upload.

After setup, run on the Ventuno Q from the hackathon checkout:

timeout 120 .venv-lm/bin/python examples/standalone-litert-lm-npu/main.py \
  --model models/gemma-4-E2B-it_Qualcomm_QCS8275.litertlm

The example also accepts --image PATH or --audio PATH with --prompt TEXT, and selects the NPU for the corresponding encoder. For audio, the example recommends a short 16 kHz mono WAV file.

Validation scope and limitations

The hackathon project documents successful text, image, and audio NPU runs with litert-lm-api==0.18.0 using short examples on one Ventuno Q. Its smoke check requires an NPU evidence: line and a PASS: local Gemma answered; backend=NPU; ... result; a generated answer alone is not evidence of NPU execution. This upload was checked for file integrity, but no new board inference or benchmark was run as part of publishing it.

The NPU context length is fixed at compilation and cannot be increased at runtime. The example limits generated output to 128 tokens; that output limit is not the model's context length. No numerical context-length or per-component quantization specification is asserted here for this artifact. The general quantization description and benchmark tables elsewhere in this card should not be treated as measurements or specifications for this particular QCS8275 file.