gemma4-llm

Gemma4 LLM

Run Google's Gemma 4 locally with vision. A fully offline AI chat app powered by Gemma 4 E2B — no cloud, no API keys, no accounts.

Downloads

Both bundles include the model, inference engine, and chat UI. Nothing else to install.

About

Gemma4 LLM bundles everything needed to run Google's Gemma 4 E2B model on your device — the model weights, inference engine, and chat UI are all included.

Gemma 4 E2B is natively multimodal — it understands both text and images out of the box. This isn't a bolt-on vision module; image understanding is built into the model architecture.

• Natively multimodal: Text and image understanding built into the model
• Fully offline: No internet connection required after install
• GPU accelerated: Uses CUDA on Windows, CPU on Android (arm64)
• Stock llama.cpp: Built on official ggml-org/llama.cpp release b8683

Model Details

Windows

1. Download and extract Gemma4-LLM.zip
2. Double-click Gemma4-LLM.exe — a native window opens with the chat UI
3. The model loads automatically on startup

The zip contains the EXE and a models/ folder side by side. Bundles CUDA runtime for GPU acceleration — falls back to CPU if no GPU is available.

Android

1. Download gemma4-llm.apk
2. Enable "Install from unknown sources" in Settings
3. Install and open — first launch extracts the model (~3 min)

Requirements:

• Android 9+ with arm64 (64-bit) processor
• 8+ GB RAM recommended
• Runs as a foreground service with status notification
• Model is bundled inside the APK (split into chunks, reassembled on first launch)

Technical Details

• Inference: llama.cpp server running locally on port 8080
• Context length: 4096 tokens (mobile), 8192 tokens (desktop)
• KV cache: q8_0 quantized for reduced memory usage
• Android: 4 threads on big cores, WebView UI
• Windows: PyWebView (Edge/Chromium), Flask backend

Credits

• Google for the Gemma 4 model family
• ggml-org/llama.cpp for the inference engine
• unsloth for GGUF quantizations