DiffusionGemma-26B-A4B-it-NVFP4-GGUF

DiffusionGemma 26B-A4B-it - NVFP4 GGUF

NVFP4 quantization of google/diffusiongemma-26B-A4B-it, a 26B parameter MoE diffusion model for text-to-image generation with 4B active parameters.

About the Model

DiffusionGemma is a block diffusion model built on top of the Gemma 4 architecture, designed for text-to-image generation.

• 26B total parameters with 4B active per token (128 experts, 8 active)
• 30-layer MoE decoder with sliding + full attention hybrid
• 27-layer vision encoder for image understanding
• Text-to-image generation - generates images from text prompts
• Block diffusion - iterative refinement approach to image generation

Quantization

This GGUF was quantized from the BF16 safetensors using llama.cpp (build 537). The source weights were converted to F16 GGUF, then quantized to NVFP4 format.

NVFP4 (NVIDIA FP4) uses 4-bit floating point quantization optimized for NVIDIA Blackwell (B-series) GPUs.

Files

Usage

llama.cpp

llama-server \
  -m diffusiongemma-26b-a4b-it-nvfp4.gguf \
  --mmproj mmproj-diffusiongemma-26b-a4b-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

Hardware Requirements

• Minimum: 16 GB VRAM for partial offload
• Recommended: 24+ GB VRAM for full GPU offload

License

Apache 2.0 - same as the base model.

Note: Vision support requires the mmproj file. The mmproj is shared between the NVFP4 and MXFP4 variants.