Gemma-4-26B-A4B-text-base

Google Gemma-4-26B-A4B-text-base (Modality-Stripped Text Backbone)

Gemma-4-26B-A4B-text-base is a clean, text-only extraction of Google's Gemma-4-26B-A4B-it Mixture-of-Experts (MoE) foundation model. By decoupling and stripping multimodal vision and audio encoders, this checkpoint provides a pure language model backbone (Gemma4ForCausalLM) in unquantized BF16.

---

⚡ Why Modality-Stripped?

Standard Gemma-4 checkpoints bundle vision transformer parameters and multimodal cross-attention layers. For dedicated language, reasoning, coding, and tool-calling deployments:

1. VRAM Reduction: Eliminates ~1.15 GB of non-language parameters and vision cache buffers.
2. Simplified Inference: Eliminates multimodal pre-processing overhead and dependencies in serving engines.
3. Pristine Weight Fidelity: The underlying text transformer parameters (model.language_model.*) are 100% identical bit-for-bit to Google's upstream release.

---

🔬 Model Profile

• Parameter Count: $26\text{B}$ total parameters ($\sim 4\text{B}$ active per token across 128 routing experts).
• Context Length: 262,144 tokens.
• Format: Standard Hugging Face safetensors compatible with transformers >= 4.40.0 and vllm >= 0.7.0.
• Vocabulary Size: 256,000 tokens.

---

⚠️ Important: Tool-Calling Parser Bugfix in vLLM

• Upstream Bug Report: vllm-project/vllm#54256
• Official Pull Request: vllm-project/vllm#54257 — [Bugfix][Parser] Support bare call: and whitespace-free channel transitions in Gemma4 parser

---

🚀 Serving with vLLM

vllm serve wyattearp/Gemma-4-26B-A4B-text-base \
  --served-model-name google/gemma-4-26B-A4B-text-base \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype bfloat16 \
  --max-model-len 262144 \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4 \
  --enable-auto-tool-choice

---

📄 License

Inherits upstream Gemma Terms of Use from Google DeepMind: Gemma Terms of Use.