Qwen3.8-Sharp-Seeded-Chat-Template

Qwen3.8 Sharp Seeded chat template

A chat template for the Qwen3.8 thinking models that cuts reasoning tokens at the default xhigh effort by roughly half and removes the runaway-thinking tail, while keeping (in this suite, slightly improving) answer quality. It is a tuned revision of peculiar-ragdoll/Qwen-Sharp-Chat-Templates, which is itself froggeric/Qwen-Fixed-Chat-Templates v22.4.1 plus a terseness system prompt. Both upstreams are Apache-2.0; so is this.

The whole change versus upstream is one sentence in the system prompt and two words in the generation prompt — the <think> block is opened with Let me instead of being left empty.

Where it was tested. All local measurements were taken self-hosting Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 on a single NVIDIA DGX Spark (GB10, 121 GiB unified memory) with vLLM, using the deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark — this file ships there as files/chat_template_sharp.jinja and is enabled with CHAT_TEMPLATE=sharp. The replication on the official Qwen3.8-27B ran against a third-party provider through OpenRouter, so the effect is not tied to that quant, that host, or that serving stack.

Every number in this card comes from a run whose raw JSON (prompts, full reasoning traces, answers, scores) is in results/. Nothing is estimated.

Why it works: the "We need answer" mode

Across every medium-effort run in this repository (906 traces on Flash-Next), 33 reasoning traces ran past 8,000 tokens. 31 of them open with the words "We need answer" — a clipped, note-taking style ("Need X. Could Y. Potential issue: Z. Good.") that spirals into dozens of consecutive "Potential issue:" paragraphs, arguing with an imagined grader about unstated requirements, and frequently never emits </think> at all. Traces that open "The user wants…" or "Let me…" of their own accord — a narrative style — ran away 0 times in 227 samples, at the same median length. (The one narrative-opener runaway was a 9.6k-token trace under the rejected The user seed; it terminated on its own.)

This is not a quirk of one quant. At xhigh effort 42 of 42 traces open "We need answer" on the local Flash-Next NVFP4 checkpoint, 42 of 42 on the official Qwen3.8-27B served by a third party, and 9 of 9 on Alibaba's closed qwen3.8-flash API. It is the trained xhigh behaviour of the Qwen3.8 line. The Flash-Next checkpoint also enters it ~40% of the time at medium; the 27B does not.

Seeding the think block with Let me makes the narrative style the only continuation the model can take. Every trace under the seeded template opens "Let me think through…" / "Let me work through…", and the runaway count across 222 seeded samples on Flash-Next (all three efforts plus the stress prompts) is zero, versus 10 in 210 for the upstream template.

What did not work

Six attempts to fix the same thing with instructions all made it worse, several dramatically. This is the part most worth reading if you edit Qwen templates yourself; details and numbers are in CHANGELOG.md.

Rules that fell out of this: never name the behaviour you want suppressed; never refer to the spec, tests, or grader as objects; never mention verification or errors in thinking guidance; do not put untrained text in the trained Reasoning effort is set to … slot; and never judge a template change on n=2 — judge on the median, the runaway count, and a 6-rep stress on prompts that are slightly under-specified.

What exactly changed

Diff against upstream v0 (versions/v0.jinja, byte-identical to the peculiar-ragdoll file):

1. In the terseness block, when thinking is on, one added sentence:

   While thinking: work the problem once, in order, making each decision the first time you meet it and building on it. Once the approach is settled, write the answer in the form the request asked for.

2. The generation prompt ends <think>\nLet me instead of <think>\n — unless the last message has role: tool, in which case it stays <think>\n (after a <tool_response>, Let me primed "…call the tool again" or an empty turn).

Everything else — effort steering phrases, high→xhigh alias, terse kwarg, tool-call formats, vision handling, the <|think_*|> in-content overrides — is upstream's, unchanged.

The seed text is part of the prompt, so it is not returned in the reasoning field by vLLM's qwen3 reasoning parser; the returned reasoning simply begins mid-sentence (" think through the problem…").

Costs and caveats

• Effort-dependent. The seed pays exactly where the model would otherwise open in the clipped style. At xhigh that is always. At medium it is ~40% of the time on Flash-Next (net −11%) and never on the 27B (net +12%, one more wrong answer in 42). If you run a checkpoint at medium that does not show the "We need answer" opener, the seed is neutral.
• Easy problems get longer. On short maths the median CoT rises 20–50%: the model narrates instead of jotting. Net token spend is still lower at every effort on Flash-Next because the tail dominates the mean.
• Answers are ~8% longer under the seeded template.
• Not a standard benchmark. The suite is 21 prompts written for this work (below), scored objectively. n = 42–126 per cell. One host, one quant, temperature 1.0. The 27B replication is 42 + 12 samples per cell on one third-party provider.
• Upstream's own caveat still applies: this template's default effort is medium (no steering line) where the checkpoint's own template defaults to xhigh; pass reasoning_effort explicitly if you depend on that.

Methodology

Everything is reproducible from bench/.

Suite (bench/tasks.py): 21 prompts, every objective task has ground truth that was brute-forced or unit-tested before any model saw it.

Runs: temperature 1.0, max_tokens 16000 (24000 for the local xhigh cells). Reasoning length = usage.completion_tokens_details.reasoning_tokens from vLLM. "Runaway" = more than 8,000 reasoning tokens. "Stress" = 6 reps of the two prompts that provoke runaways (c2 expression evaluator — leaves division-by-zero unspecified; c5 word ladder).

A/B design: one vLLM launch with --trust-request-chat-template; every version is passed per request via the chat_template field, so all versions hit identical weights, kernels, sampling and server state. bench/bench.py --template versions/v10.jinja --label v10 --effort xhigh.

Replication on the 27B: bench/bench_or.py renders the template locally with jinja2 and sends the rendered string as a raw prompt to OpenRouter's /completions, pinned to the Phala provider (which honours raw prompts; Novita rejects /completions, Io Net re-wraps the prompt). Reasoning is the text before </think>, counted with the local tokenizer.

Tool calls: bench/tooltest.py, 10 reps: does the model emit a well-formed tool_call, and after the tool result does it answer cleanly (no re-call, non-empty)? v0 10/10 and 10/10; v9 10/10 and 7/10; v10 10/10 and 10/10.

Local host: NVIDIA DGX Spark (GB10, 121 GiB unified memory), vLLM qwen38-flash-next image, Mia-AiLab/Qwen3.8-Flash-Next-NVFP4, deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark (CHAT_TEMPLATE=sharp there is this file).

Full results index

results/flash-next/ — local vLLM, Mia-AiLab/Qwen3.8-Flash-Next-NVFP4

results/qwen3.8-27b/ — qwen/qwen3.8-27b via OpenRouter → Phala, raw completions

results/qwen3.8-flash-api/probe.md — Alibaba's closed qwen3.8-flash through the chat API, 9 samples (cannot be seeded; documents that the mode is present there too).

Each JSON row has id, kind, rep, elapsed, prompt_tokens, completion_tokens, reasoning_tokens, answer_tokens, finish, score, content, and the full reasoning trace.

Usage

vLLM

vllm serve <model> --chat-template chat_template.jinja --reasoning-parser qwen3 ...

Per request: "reasoning_effort": "low" | "medium" | "high" | "xhigh" (high is an alias for xhigh). Opt out of the terseness block with "chat_template_kwargs": {"terse": false}; the seed stays. Disable thinking with "chat_template_kwargs": {"enable_thinking": false}.

LM Studio / llama.cpp: paste the file as the model's chat template. The <|think_low|> / <|think_xhigh|> / <|think_off|> in-content overrides from upstream still work as documented there; note they are template-level string triggers, not tokenizer specials.

Other Qwen3.x thinking models: untested. The seed is safe (it is two words of ordinary text) but the benefit depends on whether the model opens in the clipped style at your effort level. Check five traces before assuming.

Attribution

• Template base: froggeric/Qwen-Fixed-Chat-Templates v22.4.1 (Apache-2.0).
• Terseness block and the Sharp variant: peculiar-ragdoll/Qwen-Sharp-Chat-Templates (Apache-2.0). versions/v0.jinja is that file verbatim.
• Tuning, suite, measurements and this card: mdrbal83, September 2026, on the DGX Spark deployment kit MiaAI-Lab/Qwen3.8-Flash-Next-Single-DGX-Spark.
• Checkpoint used locally: Mia-AiLab/Qwen3.8-Flash-Next-NVFP4.

If you find a Qwen3.x checkpoint where the seed hurts, or a seed text that does better than Let me, open a discussion — the suite makes it a 30-minute check.