gemma-4-26B-A4B-mlx

gemma-4-26B-A4B-mlx (BF16)

This model was converted to MLX format from google/gemma-4-26B-A4B using mlx_lm.convert at full BF16 precision (no quantization). Refer to the original model card for more details on the model.

Model Details

• Base model: google/gemma-4-26B-A4B (base, non-instruction-tuned)
• Architecture: Mixture-of-Experts (MoE), 25.2B total parameters, ~4B active per token
• Format: MLX (Apple Silicon)
• Precision: BF16 (full precision, no quantization)
• License: Gemma Terms of Use

Installation

pip install -U mlx-lm

Usage

Command line

mlx_lm.generate \
  --model your-username/gemma-4-26B-A4B-mlx \
  --prompt "The future of AI is" \
  --max-tokens 256

Python API

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("your-username/gemma-4-26B-A4B-mlx")

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=64)

response = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="The future of AI is",
    max_tokens=256,
    sampler=sampler,
    verbose=True,
)
print(response["text"])

Note: In current mlx-lm versions, sampling parameters (temperature, top_p, top_k) must be passed via a sampler object created with make_sampler(), not as direct keyword arguments to generate().

Conversion details

This model was converted with the following command:

mlx_lm.convert \
  --hf-path google/gemma-4-26B-A4B \
  --mlx-path models/gemma-4-26B-A4B-mlx

Hardware requirements

• Apple Silicon Mac (M1/M2/M3/M4)
• Minimum 64GB unified memory (BF16 full precision, no quantization)
• The unquantized 26B MoE model requires significantly more RAM than 4-bit/8-bit variants

Limitations

• This is the base model (not instruction-tuned). For chat-style interactions, use google/gemma-4-26B-A4B-it as the source instead.
• MLX text-only inference via mlx-lm. For multimodal (image+text) support, use mlx-vlm with an MLX-converted instruction-tuned variant.
• Due to the full-precision weights, memory usage will be substantially higher than quantized versions. Use a 4-bit or 8-bit variant if your Mac has less than 64GB RAM.