gemma4-12B-it-DFlash-GGUF

Gemma 4 12B IT DFlash GGUF

GGUF quantizations of z-lab DFlash draft model for Gemma 4 12B IT.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

gemma4-12B-it-DFlash

Paper | GitHub | Blog

DFlash is a speculative decoding method that uses a lightweight block diffusion model to draft multiple tokens in parallel. This is the drafter model, which must be paired with google/gemma-4-12b-it. It is not a standalone Gemma 4 12B model.

<div align="center"> <img src="assets/dflash_system.png" alt="DFlash Architecture" width="85%"> </div>

The original z-lab checkpoint does not include a model card. The usage instructions below apply to these GGUF files in BeeLlama.cpp.

Quantizations

All quantizations are generated directly from a BF16 conversion of the original checkpoint using BeeLlama.cpp.

Quick Start

Installation

Build or download BeeLlama.cpp. Download a GGUF of the Gemma 4 12B IT target model separately, along with one draft quantization from this repository.

Launch Server

Run from the directory containing your model files, replacing target.gguf with the path to your Gemma 4 12B IT GGUF:

llama-server -m target.gguf \
  --spec-type draft-dflash \
  --spec-draft-model gemma4-12B-it-DFlash-Q8_0.gguf \
  --spec-draft-n-max 16 \
  --flash-attn on \
  --port 8080

The draft checkpoint is trained with a block size of 16. Use draft-dflash, not the removed bare dflash alias, in current BeeLlama.cpp.

Usage

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="target.gguf",
    messages=[{"role": "user", "content": "Write a quicksort in Python."}],
    max_tokens=4096,
    temperature=0.0,
)
print(response.choices[0].message.content)

Benchmark Results

No benchmark results for these Gemma 4 12B IT GGUF quantizations are published here. Results from other DFlash draft models are not measurements of this checkpoint.

Citation

If you find DFlash useful, please cite the original work. To share feedback on DFlash or request new model support, use DFlash Feedback.

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}