Qwen3.5-9B-DFlash-GGUF

Qwen3.5-9B-DFlash-GGUF

GGUF conversion of z-lab/Qwen3.5-9B-DFlash, the block-diffusion speculative-decoding draft model for Qwen3.5-9B.

This is not a standalone model. It only does anything paired with the target model in a speculative-decoding server.

Requirements

Upstream llama.cpp — DFlash support was merged in #22105 (2026-06-28). No fork needed. Older builds fail with unknown model architecture: 'dflash'.

Pair it with a Qwen3.5-9B target GGUF (e.g. unsloth/Qwen3.5-9B-GGUF):

llama-server \
  -m Qwen3.5-9B-UD-Q4_K_XL.gguf \
  --spec-draft-model qwen3.5-9b-dflash-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 4

Tune --spec-draft-n-max — it matters a lot

The default draft window is far too long for this model. Measured on an RTX 5090, Qwen3.5-9B target at Q4_K_XL, 32k context, q8_0 KV, same 400-token coding prompt:

At 15 the draft proposes long runs and ~73% get discarded. At 4 it stays inside the block the diffusion draft actually predicts well, and throughput is ~1.8x the no-speculation baseline.

Also measured on an Intel Arc Pro B50 (SYCL): same ~27% acceptance at n-max 15, so the effect is backend-independent.

Expected warnings

On load you will see, on a perfectly healthy run:

llama_init_from_model: failed to initialize the context: dflash requires ctx_other to be set
  (this warning is normal during memory fitting)
srv load_model: [spec] failed to measure draft model memory: failed to create llama_context from model

DFlash drafts carry no token_embd / output tensors — they borrow the target's. The pre-fit measurement pass loads the draft standalone, before ctx_other is wired up in common/speculative.cpp, so it always throws. Harmless; the server continues and loads the draft normally. The only cost is that the draft's memory is not counted during auto-fit, which can matter on a tight VRAM budget if you let --ctx-size or -ngl be auto-selected (see --fit-target, --fit off).

Conversion

The draft has no tokenizer of its own, so conversion requires --target-model-dir pointing at the target's HF repo (only its tokenizer and config are read, not the weights):

python convert_hf_to_gguf.py z-lab/Qwen3.5-9B-DFlash \
  --outtype bf16 \
  --target-model-dir Qwen/Qwen3.5-9B \
  --outfile qwen3.5-9b-dflash-BF16.gguf

llama-quantize qwen3.5-9b-dflash-BF16.gguf qwen3.5-9b-dflash-Q4_K_M.gguf Q4_K_M

Resulting metadata: general.architecture = dflash, block_size = 16, target_layers = [2, 6, 10, 14, 18, 22, 26, 30], sliding window 4096.

License & credit

Apache-2.0, inherited from the source model. All model work is by Z-Lab and Modal; this repo only contains a format conversion and quantization.