Qwopus3.5-9B-Coder-DFlash-GGUF

Qwopus3.5-9B-Coder-DFlash-GGUF

DFlash draft model for Jackrong/Qwopus3.5-9B-Coder, for use with upstream llama.cpp speculative decoding (--spec-type draft-dflash, merged in #22105).

This is z-lab/Qwen3.5-9B-DFlash converted with --target-model-dir pointing at the Qwopus tokenizer. Qwopus extends the Qwen3.5 tokenizer with 7 added tokens (ids 248070 to 248076), so drafts converted against the base Qwen3.5-9B tokenizer do not pass the vocab compatibility check for this target. The draft GGUF carries no token embeddings or lm_head; llama.cpp shares the target model's at runtime, so the draft matches whatever Qwopus quant you serve.

Conversion and launch scripts: https://github.com/Gaurav-Gosain/qwopus-dflash

<video controls src="https://huggingface.co/GauravGosain/Qwopus3.5-9B-Coder-DFlash-GGUF/resolve/main/demo.mp4"></video>

The demo replays two real captured token streams applying the same edit to a Go file (same prompt, temperature 0): baseline left, DFlash right.

Usage

llama-server \
  -hf Jackrong/Qwopus3.5-9B-Coder-GGUF:Q3_K_M \
  -hfd GauravGosain/Qwopus3.5-9B-Coder-DFlash-GGUF:Q4_K_M \
  --no-mmproj \
  --spec-type draft-dflash --spec-draft-n-max 15 \
  -fa on --jinja -c 4096 -ctk q8_0 -ctv q8_0 -ctxcp 2 -fitt 256

Pass -hf and -hfd together (local -m plus -hfd currently fails to resolve the draft) and keep --no-mmproj (the target repo ships a 921 MB vision projector).

Qwen3.5 is a hybrid linear-attention architecture; keep -ctxcp low because each context checkpoint stores the full recurrent state (about 100 MB).

Files

Measured (RTX 3070 8 GB, target Q3_K_M, temp 0, back to back, both fully on GPU)

The speedup tracks output predictability; editing existing code is the best case (mean draft length 13.7 of 15). Freeform prose drops to about 0.15 acceptance, still a net win. Both models plus buffers need about 6.5 GB free VRAM and a low fit margin (-fitt 256); if the target spills layers to CPU, speculation goes net-negative.

Also works on Apple Silicon via dflash-mlx: on an M3 Pro 18 GB, code editing goes 28 to 55 tok/s (1.95x) and fresh generation 28 to 42 tok/s (1.49x) with --draft-quant w4 --block-tokens 8. Setup scripts in the GitHub repo.