FranQwen3.8-27B-NVFP4-DFlash2

FranQwen3.8-27B-NVFP4-DFlash2

A Frankenstein Qwen3.8-27B. One model's body, another model's head, and a third model's drafter. The name is honest about what it is.

The point of the stitching is speed. Stock NVFP4 checkpoints can't run DFlash2 speculative decoding at all.

Tested on DGX Sparks

Abliterated. Refusal behaviour was removed upstream and this doesn't put it back. No safety tuning. See Safety.

The parts

Three donors, each contributing one thing:

Why swap the head

DFlash2 needs an unquantized lm_head. NVFP4 checkpoints quantize theirs, so vLLM simply refuses to start with a DFlash2 config.

You might think you could just convert the head back to BF16. That doesn't work. Converting back gives you the same rounding damage in twice the storage, so the drafter is still guessing against a degraded output distribution. What's needed is a head that was never quantized in the first place, which is what sakamaki's checkpoint provides.

Is that safe to do?

Checked before merging, not after. The donated head and the body's own head agree to a cosine similarity of 0.999647, with the leftover difference sitting exactly at quantization-rounding size. If abliteration had modified the head you'd see a much bigger, structured difference.

The quality results below confirm it behaviourally too.

Speed

Measured with llama-benchy, the tool Spark Arena. Hardware tested is NVIDIA DGX Spark (GB10).

One node

Two nodes, TP=2 over ConnectX-7

Going from one node to two gives about 1.5x on single-stream generation, and KV capacity grows from 900k to 3.8M tokens.

For comparison, the best published two-node Qwen3.8-27B result on Spark Arena for the same benchmark is 40.09 tok/s at concurrency 1, and 115.04 at concurrency 5.

A few honest notes. Single-stream speed bounces around by roughly 12% run to run, so use an average of several runs rather than your best one. Speed also depends a lot on the workload: code prompts hit 55.3 tok/s on one node because the drafter guesses code far better than prose.

Quality

Run with lm-evaluation-harness on full task sets, speculative decoding off:

Two things worth pulling out.

The graft is free. Against the unmodified body, every difference is smaller than the measurement error. Three metrics up, two down, one identical to four decimal places.

It takes the head and nothing else. Look at the head donor's column: that checkpoint is 15 points worse at gsm8k. If grafting dragged across the donor's characteristics, this model's gsm8k would have sagged toward 0.6156. It didn't move at all.

It also scores 44/46 on a private 46-question reasoning set, tying the best of every Qwen3.8-27B checkpoint tested. Vision still works, including with DFlash2 active.

An accidental finding: abliteration makes the drafter's job easier

Worth writing down because it wasn't expected.

The same BF16 head was grafted onto a non-abliterated NVFP4 Qwen3.8-27B and benchmarked identically. Same drafter, same settings, same everything except the body.

Generation is 11–24% faster on the abliterated body, while prompt processing is identical to within a rounding error. Identical prefill and different decode points at one thing: the drafter's suggestions get accepted more often. The abliterated model is simply more predictable to guess.

Quality is the same either way, so if you want a non-abliterated build, the same graft works fine on a stock body and costs you roughly 20% throughput.

Running it

Needs vLLM with DFlash2 support (PR #52816, in mainline since 2026-08-21) and the drafter downloaded separately. Only tested on GB10.

vllm serve <this-model> \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.6 \
  --max-model-len 262144 \
  --max-num-batched-tokens 16384 \
  --max-num-seqs 8 \
  --trust-remote-code \
  --enable-prefix-caching \
  --speculative-config '{"method":"dflash","model":"z-lab/Qwen3.8-27B-DFlash2","num_speculative_tokens":7}' \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml

Use 7 speculative tokens, not 8. Eight is accepted but gave no measurable gain and destabilised things at higher concurrency.

Give it max_tokens of at least 8192 for real work. It's a reasoning model, and with a small budget the thinking eats the whole allowance and you get empty output back.

Safety

This model is abliterated. Refusal behaviour was removed upstream, and swapping the head doesn't restore it, because the modified weights are in the body.

It will go along with requests that the official Qwen3.8-27B declines, including harmful ones. It has had no safety tuning, and scoring well on reasoning benchmarks says nothing about how it behaves. If you're putting this anywhere users can reach it, add your own filtering and evaluate it for your own use case.

Credits

• Qwen/Qwen3.8-27B, the base model
• orcarouter/Qwen3.8-27B-Uncensored-NVFP4, for the abliterated NVFP4 body
• sakamaki, for the BF16 lm_head
• z-lab/dflash, for DFlash2 and the drafter
• eugr/llama-benchy and Spark Arena, for the benchmark harness

Everything upstream is Apache-2.0, and so is this.