Qwen3.8-27B-Leimroth3

Qwen3.8-27B-Leimroth3

Model id leimroth-lab/Qwen3.8-27B-Leimroth3.

An abliterated build of Qwen/Qwen3.8-27B: a single refusal direction is orthogonalised out of the language-layer writer matrices at lambda 1.5, so the model declines far less often. The vision tower is left untouched.

By Scott Leimroth · scottleimroth.com · @ScottLeimroth

Update, 2026-09-29: full lever study, and a tested serving recipe

Over 28 and 29 September I ran every serving lever on this model, one at a time, on a single DGX Spark (GB10, 128 GB unified memory): thinking level, precision, drafter, draft budget, engine mode, attention backend, context and memory, plus an unedited build of the same base as a control. Every number below comes from that study, every figure was read back from its raw result file, and the setup behind each one is stated. The earlier capability and speed sections are superseded and the older correction (8 September) is kept below because it is still right about the method.

The short version: set the thinking level explicitly, and set it to medium. Left at the template default, this model family thinks until the output budget is gone and returns nothing. That one setting matters more than any other on this page.

Correction, 2026-09-08

Three things on this page were wrong or unsupported, found while re-auditing my own measurements. They are corrected below rather than quietly edited out.

1. At lambda 1.5 the refusal direction is not removed, it is inverted. The edit is W <- W - lambda * r (r^T W), so the component along r is multiplied by (1 - lambda). At lambda 1.0 that is exact removal. At the lambda 1.5 used here it is -0.5: the component is re-injected with its sign flipped at half magnitude. This build therefore does not merely have refusal ablated; it is pushed some way into the opposite. Language elsewhere on this page describing the direction as "removed" or "projected out" is accurate only for lambda <= 1, and the lambda was not previously stated at all. Verified algebraically and numerically.

2. The original compliance figure (90.6%) was measured on 48-token continuations, which is too short for the rubric. It is retained below as history only. The compliance figures that matter are the automatic ones at an 8,192-token budget in the Evaluation section.

3. "Capabilities left intact" was not established at the time. It now has a like-for-like control; see Capability.

A related claim I made publicly, that abliteration costs multi-turn instruction-following, has been withdrawn at the source: https://github.com/dreamfast/abliterlitics/issues/1

What this is

Standard abliteration (refusal-direction orthogonalisation). A single refusal direction is estimated from the difference in hidden states between harmful and harmless prompts, then projected out of the model's writer matrices in the language layers. The vision tower and projector are preserved, so the model is still multimodal.

Method

• Direction: mean-difference of hidden states, harmful minus harmless prompts, over a held-out probe set.
• Layer selection: direction layer chosen by a KL tie-break; layer 30 selected.
• Edit: W <- W - lambda * r (r^T W) over 112 writer matrices from layer 8 upward, lambda = 1.5; language layers only; vision tower and multi-token-prediction tensors untouched. Note that lambda > 1 inverts the component rather than removing it (see the correction above).
• Format: full multimodal checkpoint, BF16.

The serving recipe, tested

Everything below was measured on this recipe unless it says otherwise. It is the setting the study found best, and it is what I run in production:

• NVFP4 export of this checkpoint (made with NVIDIA's Model Optimizer; the export itself is not published), served with SGLang on one GB10.
• Thinking level medium, set explicitly in the chat template arguments (reasoning_effort: medium). Do not leave it at the default.
• DFlash2 speculative decoding with an NVFP4 build of z-lab/Qwen3.8-27B-DFlash2 at a draft budget of 8, which is that head's block size and therefore its maximum.
• Context 262,144, memory fraction 0.72, CUDA graphs on, flashinfer attention, FP8 KV cache, temperature 0 for all measurements.

Why medium. On a 60-item agent benchmark (tool use, structured output, multi-turn, sequencing, long context; 8,192-token output budget):

Left at the template default with no level set, a sibling build of the same base returned empty answers on 16 of 63 plain prompts and 21 of 30 severe ones at an 8,192-token budget: the model reasons until the budget is exhausted and never answers. At medium, zero empties across 180 agent tasks and 186 compliance prompts. Inside an agent context with tools the same sibling ran away on 1 of 63, so the failure is worst on bare prompts.

Capability, measured against an unedited control

Same benchmark, same recipe, same box, one thing changed:

The four items this model misses are all multi-turn, and the unedited base misses the same four, so they are not caused by the edit. Two boots of one setup can differ by two to four items on this suite, so treat single-item gaps as noise; the four-item gap here is one boot of the control against three of the edit, and I make no stronger claim than "no sign that the edit costs accuracy".

Precision. NVFP4 56 / 60 (the recipe above). At an earlier, smaller-context setting, a BF16 serve scored 55 and an FP8 serve 54, with likelihood on a fixed corpus of 0.5245, 0.5253 and 0.5314 bits per character for BF16, FP8 and NVFP4. All three sit inside the run-to-run band. Four-bit costs nothing measurable here.

Speed

Decode speed on this recipe, single stream, temperature 0, idle box, three workloads of 512 tokens each, 9 runs after warm-up:

So speculation is worth 3.6x here. Per-agent speed with several agents at once: 29.8 tok/s each with 2 streams, 25.7 each with 4 (102.7 aggregate). Whole-serve memory footprint 90.9 GiB; KV pool 1,014,953 tokens at the 262,144 context.

What the other levers did, so you do not have to repeat them:

• Drafter. The NVFP4 head above is the fastest. A BF16 build of the same head is 5% slower (41.7 tok/s) for the same accuracy; a head re-calibrated on this checkpoint's own outputs is 1.5% slower (43.2). On the unedited base the same head achieves 3.8x, so the edit costs a few percent of the drafter's benefit, not raw decode speed (12.14 base against 12.18 edited without a drafter).
• Draft budget. 8 is the head's block size. SGLang accepts a larger budget with only a warning and runs anyway; drafts past the block are meaningless. Do not raise it.
• CUDA graphs. On: 43.9 tok/s. Off (eager): 42.6, with all 60 benchmark outcomes identical. Keep them on.
• Attention backend. flashinfer works. Triton cannot serve with DFlash on the SGLang build I use: the speculative worker crashes on the first batch ('DFlashDraftInputV2' object has no attribute 'nxt_kv_lens_cpu'). Without the drafter, triton serves at the same 12.2 tok/s and scored 54 / 60.
• Boot-to-boot noise, worth knowing before you compare anything: two boots of one configuration differed by 6.8% in drafter-on speed on the base, and by two to four agent items on this suite under a change that cannot alter outputs (draft length). Differences smaller than that between single boots are not findings.

Evaluation

These are my own measurements, shown so you can reproduce them, not an independent validation.

Compliance, automatic, 8,192-token budget, temperature 0, medium thinking, on a pinned set of 63 harmful requests and 7 legitimate ones, bare (no system prompt) and again inside a real agent context with tools, plus a 30-item severe subset in both forms. Every reply is counted in one of three buckets: complied, refused, or empty (the model reasoned until the budget ran out and said nothing). Empties are never folded into either of the other two.

The three-way count is the part worth copying: a pass/fail refusal number hides the model that answers and says nothing, and hides the model that says nothing at all.

Historical figure, superseded: 90.6% comply, 9.4% hedge, 0 refusal on a 64-prompt set with hand labels on 48-token continuations (too short for its rubric; see the 8 September correction).

Coherence: mean KL(base ‖ edited) over held-out harmless texts stayed well inside budget; benign generations near-identical to the base.

Reading images: read this before you serve it

Serve image requests with the thinking step off. On a 19-item screenshot and photo set at this recipe, with thinking off it answered 19 of 19 and scored 7.1 of 10 against a pre-committed answer key. At an 8,192-token budget, medium thinking also answers all 19 (the older empty-answer failure was a 2,000-token budget), but it took 39% longer and found fewer of the key's facts (162 against 168 of 183), so thinking buys nothing on images.

Message order changes the answers materially: text before image and image before text are not interchangeable, so keep the order fixed across anything you compare. When it errs it tends to state a wrong specific confidently rather than miss text; if a confidently wrong number or name is expensive in your application, that is the failure mode to test for.

Sharing one Spark with other work

If Whisper and a video-rendering stack have to live on the same box, this model fits at memory fraction 0.40 with a KV pool of 117,505 tokens, which is less than one full 262,144-token request: a real 15-second render succeeded beside it with free memory never below 17 GiB. At 0.44 and above the render guard fails. On a box of its own, use the recipe above.

Limitations

• The compliance rows above are single boots of one instrument set; re-measure for your own use.
• Capability is compared against one boot of the unedited base; the edit shows no accuracy cost on this suite, and that is the extent of the claim.
• Abliteration removes a safety behaviour, and at lambda 1.5 pushes past removal into the opposite. See Intended use.
• The NVFP4 export and the NVFP4 drafter build used for the speed figures are not published; with this BF16 checkpoint, use z-lab's original drafter and expect the BF16-head figures above.

Intended use and responsible use

This model has had a refusal behaviour removed and will answer prompts a stock instruct model would decline. It is released for research into alignment, refusal mechanisms, and red-teaming. You are responsible for how you use it and for complying with the base model's license and applicable law. Do not deploy it in a user-facing setting without your own safety layer.

License and credits

Derived from Qwen/Qwen3.8-27B, released by Qwen under Apache 2.0, which permits redistribution of modified models. This derivative is released under the same license, with attribution to the base model above.

The method follows the public refusal-direction / orthogonalisation line of work on abliteration; this is an application of that published technique, not a new one.

Benchmarks for this model alongside the others measured on the same hardware, with the exact serving config behind every number: https://scottleimroth.com/ai-technology/spark-benchmarks

Serving stacks: vLLM and SGLang, including the community work that got both running on GB10 / sm_121, and NVIDIA TensorRT Model Optimizer for the 4-bit export recipe.