Qwen3.6-35B-A3B-DFlash-GGUF

Qwen3.6-35B-A3B-DFlash-GGUF

GGUF conversion of z-lab/Qwen3.6-35B-A3B-DFlash for llama.cpp.

This is a DFlash draft model, not a standalone language model. It must be used together with a compatible Qwen3.6-35B-A3B target model.

Model

• Base model: z-lab/Qwen3.6-35B-A3B-DFlash
• Target model: Qwen/Qwen3.6-35B-A3B
• Format: GGUF
• Quantization: Q4_K_M

Compatibility

Requires a recent version of llama.cpp with DFlash support.

Tested with:

• llama.cpp b9831 or newer

Usage

Example:

llama-server \
  -m Qwen3.6-35B-A3B.gguf \
  -md Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 3

or

llama-cli \
  -m Qwen3.6-35B-A3B.gguf \
  -md Qwen3.6-35B-A3B-DFlash-Q4_K_M.gguf \
  --spec-type draft-dflash

Conversion

Converted from the original Hugging Face model using the latest convert_hf_to_gguf.py.

No model weights were modified.

Benchmark (RTX 5070 Ti)

Benchmarked using mtp-bench.

max 2

python mtp-bench.py
  code_python        pred= 192 draft= 135 acc= 123 rate=0.911 tok/s=161.0
  code_cpp           pred=  54 draft=  36 acc=  36 rate=1.000 tok/s=170.7
  explain_concept    pred= 192 draft= 194 acc=  94 rate=0.484 tok/s=119.5
  summarize          pred=  48 draft=  36 acc=  29 rate=0.806 tok/s=154.1
  qa_factual         pred= 192 draft= 144 acc= 118 rate=0.819 tok/s=156.5
  translation        pred=  17 draft=  20 acc=   7 rate=0.350 tok/s=95.3
  creative_short     pred=  36 draft=  40 acc=  16 rate=0.400 tok/s=107.1
  stepwise_math      pred= 192 draft= 138 acc= 122 rate=0.884 tok/s=165.9
  json_output        pred= 192 draft= 130 acc= 125 rate=0.962 tok/s=166.8
  long_reasoning     pred= 192 draft= 155 acc= 112 rate=0.723 tok/s=139.8
  repeat_pattern     pred= 192 draft= 127 acc= 127 rate=1.000 tok/s=169.2
  code_completion    pred= 106 draft=  72 acc=  69 rate=0.958 tok/s=153.9
  long_code_review   pred= 192 draft= 189 acc=  95 rate=0.503 tok/s=110.5

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1797,
  "total_draft": 1416,
  "total_draft_accepted": 1073,
  "aggregate_accept_rate": 0.7578,
  "wall_s_total": 15.15
}

max 3

python mtp-bench.py
  code_python        pred= 192 draft= 154 acc= 139 rate=0.903 tok/s=180.4
  code_cpp           pred=  54 draft=  42 acc=  39 rate=0.929 tok/s=192.3
  explain_concept    pred= 192 draft= 240 acc= 110 rate=0.458 tok/s=127.1
  summarize          pred=  48 draft=  45 acc=  32 rate=0.711 tok/s=160.8
  qa_factual         pred= 192 draft= 176 acc= 132 rate=0.750 tok/s=170.7
  translation        pred=  17 draft=  30 acc=   8 rate=0.267 tok/s=82.8
  creative_short     pred=  36 draft=  60 acc=  17 rate=0.283 tok/s=94.0
  stepwise_math      pred= 192 draft= 172 acc= 133 rate=0.773 tok/s=172.2
  json_output        pred= 192 draft= 153 acc= 140 rate=0.915 tok/s=191.7
  long_reasoning     pred= 192 draft= 195 acc= 126 rate=0.646 tok/s=153.4
  repeat_pattern     pred= 192 draft= 144 acc= 143 rate=0.993 tok/s=205.0
  code_completion    pred= 101 draft=  81 acc=  73 rate=0.901 tok/s=192.4
  long_code_review   pred= 192 draft= 260 acc= 103 rate=0.396 tok/s=114.5

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1792,
  "total_draft": 1752,
  "total_draft_accepted": 1195,
  "aggregate_accept_rate": 0.6821,
  "wall_s_total": 13.75
}

max 4

python mtp-bench.py
  code_python        pred= 192 draft= 172 acc= 147 rate=0.855 tok/s=192.9
  code_cpp           pred=  54 draft=  48 acc=  42 rate=0.875 tok/s=211.4
  explain_concept    pred= 192 draft= 323 acc= 110 rate=0.341 tok/s=116.4
  summarize          pred=  48 draft=  60 acc=  34 rate=0.567 tok/s=151.4
  qa_factual         pred= 192 draft= 221 acc= 135 rate=0.611 tok/s=168.7
  translation        pred=  17 draft=  40 acc=   8 rate=0.200 tok/s=75.8
  creative_short     pred=  36 draft=  72 acc=  19 rate=0.264 tok/s=95.2
  stepwise_math      pred= 192 draft= 185 acc= 144 rate=0.778 tok/s=198.5
  json_output        pred= 192 draft= 168 acc= 148 rate=0.881 tok/s=210.1
  long_reasoning     pred= 192 draft= 221 acc= 135 rate=0.611 tok/s=164.3
  repeat_pattern     pred= 192 draft= 152 acc= 152 rate=1.000 tok/s=235.1
  code_completion    pred= 106 draft=  88 acc=  83 rate=0.943 tok/s=228.1
  long_code_review   pred= 192 draft= 371 acc=  96 rate=0.259 tok/s=98.3

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1797,
  "total_draft": 2121,
  "total_draft_accepted": 1253,
  "aggregate_accept_rate": 0.5908,
  "wall_s_total": 13.7
}

max 5

python mtp-bench.py
  code_python        pred= 192 draft= 185 acc= 154 rate=0.832 tok/s=212.1
  code_cpp           pred=  54 draft=  55 acc=  44 rate=0.800 tok/s=211.2
  explain_concept    pred= 192 draft= 388 acc= 113 rate=0.291 tok/s=113.1
  summarize          pred=  48 draft=  60 acc=  37 rate=0.617 tok/s=171.9
  qa_factual         pred= 192 draft= 255 acc= 139 rate=0.545 tok/s=168.1
  translation        pred=  17 draft=  55 acc=   7 rate=0.127 tok/s=68.8
  creative_short     pred=  36 draft=  85 acc=  20 rate=0.235 tok/s=94.2
  stepwise_math      pred= 192 draft= 219 acc= 147 rate=0.671 tok/s=194.1
  json_output        pred= 192 draft= 178 acc= 155 rate=0.871 tok/s=226.8
  long_reasoning     pred= 192 draft= 260 acc= 138 rate=0.531 tok/s=160.3
  repeat_pattern     pred= 192 draft= 160 acc= 159 rate=0.994 tok/s=263.1
  code_completion    pred= 106 draft=  90 acc=  87 rate=0.967 tok/s=252.4
  long_code_review   pred= 192 draft= 442 acc= 100 rate=0.226 tok/s=95.1

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1797,
  "total_draft": 2432,
  "total_draft_accepted": 1300,
  "aggregate_accept_rate": 0.5345,
  "wall_s_total": 13.49
}

Observations

• n_max = 2 provides the highest acceptance rate.
• n_max = 3 provides the best balance between throughput and acceptance rate.
• n_max = 4/5 may improve peak throughput slightly in some low-entropy tasks such as code completion, JSON, and repetitive patterns, but the overall wall time improves only marginally while acceptance rate drops noticeably.

Notes

This repository contains only the DFlash draft model.

A compatible Qwen3.6-35B-A3B GGUF target model is required for speculative decoding.

Credits

• z-lab — Original DFlash model
• Qwen Team — Qwen3.6-35B-A3B
• ggml-org/llama.cpp — GGUF format and DFlash inference implementation

License

This repository contains a converted GGUF version of the original DFlash draft model.

All original licenses, usage restrictions, and intellectual property remain with the upstream authors. Please refer to the original repositories for complete licensing information.