Gemma-4-Waldwicht-Juengling

<div align="center"> <img src=https://github.com/kyr0/waldwicht/raw/main/waldwicht_banner.png> <br> Jüngling - MLX Edition - 3.86 GB </div>

Overview

Waldwicht-Juengling is the highest-quality quantization of Gemma 4 E2B (2.3B effective parameters), using uniform 5-bit g64 precision. At 3.86 GB it achieves near-BF16 quality across all task categories - the result of a 50+ configuration ablation study on Apple Silicon.

This is the quality-first variant: when you have the memory budget, Juengling delivers the best scores across code generation, multi-step reasoning, translation, and creative writing. Choose this when quality matters more than size.

Part of the Waldwicht family: Winzling (2.96 GB) - Sproessling (3.17 GB) - Juengling (3.86 GB)

Quick Start

For a quick test, you can install the a current version of mlx-lm from PyPI and run mlx_lm.generate with the model name and a prompt:

pip install mlx-lm
python -m mlx_lm.generate --model kyr0/Gemma-4-Waldwicht-Juengling --prompt "Explain quantum tunneling in simple terms."

However, we recommend using the Waldwicht Inference Server.

Waldwicht Inference Server

Together with this model release, we are also launching the Waldwicht Inference Server, a high-performance server optimized for running our quantized models on Apple Silicon. The server is open-source and available on GitHub: Waldwicht Inference Server

Quantization Specification

Uniform 5-bit affine quantization with group size 64. No transforms, no rotation, no calibration data - standard MLX mx.quantize.

Key Metrics

Benchmark Results by Category

Evaluated on a 20-prompt diverse benchmark covering 6 task categories (correctness / completion / reasoning hygiene):

Highlights

• Code generation achieves near-perfect 9.8 correctness across all 4 code prompts
• Multi-step reasoning scores 9.2 correctness - up from 7.0 at uniform 4-bit, the largest quality jump between bit-widths
• Translation scores 9.0 correctness across all 4 translation prompts
• Creative writing scores perfect 10.0/10.0/10.0 across all 4 prompts
• Reasoning quality jumps sharply at 5-bit. Multi-step reasoning goes from 7.0 (uniform 4-bit) to 9.2 (uniform 5-bit) correctness

Comparison with Other Variants

Why Uniform 5-bit?

Our ablation study tested 50+ configurations including layer-level mixed-precision, component-level mixed-precision, rotation, and various group sizes. Key findings relevant to this variant:

1. Uniform beats layer-level mixed-precision for E2B. Unlike the larger E4B model where three-tier mixed-precision was essential, E2B's smaller size and flatter layer sensitivity profile makes uniform quantization optimal at each bit-depth.

2. Group size g64 is Pareto-optimal. E2B's narrow hidden dimension (1536) benefits from finer granularity. g128 causes catastrophic failure at lower bit-widths; g32 passes but wastes +0.64 GB. g64 maximizes quality per GB.

3. 5-bit is the quality ceiling. Going from 4-bit to 5-bit delivers the largest quality jumps in reasoning (+2.2 correctness), translation (+0.2), and code gen (+0.6). Going to 6-bit adds minimal improvement at +0.64 GB cost.

4. No transforms needed. Rotation was tested and provides no benefit for E2B uniform quantization. Standard mx.quantize is sufficient.

When to Choose Juengling vs Smaller Variants

Research Paper

Full details: Pushing Below 3 GB: Component-Level Mixed-Precision Quantization for Gemma 4 E2B on Apple Silicon (Homberg, April 2026). 50+ configurations, 4 experimental phases, 20-prompt diverse benchmark with LLM-judge scoring.

<div align="center"> <img src=https://github.com/kyr0/waldwicht/raw/main/waldwicht_artwork.png> </div>

Base model

For the base model documentation, see Gemma 4 E2B - the smallest dense model in the Gemma 4 family, with 2.3B effective parameters and a 128K token context window.

<div align="center"> <img src=https://ai.google.dev/gemma/images/gemma4_banner.png> </div>