Downloads · 30 days
297
26% of all-time downloads
OliviaRossi/Improved-Chat-Template-for-Qwen-3.x
Improved-Chat-Template-for-Qwen-3.x is a text generation model from OliviaRossi. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A single, production-hardened Jinja2 chat template for Qwen 2.5 / Qwen 3.x / QwQ and hybrid reasoning merges (KAT-Coder · Qwopus · AgentWorld · Ornith), built for three workloads at once:
Downloads · 30 days
297
26% of all-time downloads
All-time downloads
1.2K
Public
Repo size
—
Likes
14
Trending 2
Click a slice to open those files.
.jinja134 KB · 50%
From the Hugging Face model README
A single, production-hardened Jinja2 chat template for Qwen 2.5 / Qwen 3.x / QwQ and hybrid reasoning merges (KAT-Coder · Qwopus · AgentWorld · Ornith), built for three workloads at once:
| Posture | What it optimises for |
|---|---|
| Agent | Multi-turn tool loops, parallel calls, lossless call history, anti-loop circuit breaker |
| Coding | Agent + verification discipline (read-before-edit, minimal diffs, verify before claiming success) |
| Creative | Long-form prose: no agentic scaffolding, no terse pressure, byte-exact continuation |
The posture is selected automatically (tools present → agent) or explicitly (task_mode="creative"), or switched mid-conversation with an in-prompt directive.
chat_template.jinjaqwen-hybrid-agentic-v4.1<|im_start|> / <|im_end|>, Qwen3-Coder XML or Qwen JSON tool calls, <tool_response> blocks, no BOS.v4.1 is an audit release. It fixes two regressions introduced in v4.0 and several latent defects inherited from v3, without changing the on-the-wire prompt format. If you are on v3, read Migrating from v3.
| # | Issue | Fix |
|---|---|---|
| 1 | No-think prefill emitted <think>\n\n</think>\n (v4.0 regression) | Restored canonical <think>\n\n</think>\n\n |
| 2 | XML <IMPORTANT> rule lost its tag literal (v4.0 regression) | Restored …inside the <think></think> block. |
| 3 | Silent history corruption: any literal </think> in prose was treated as a reasoning boundary; with pruning enabled the preceding answer text was deleted from context | Only the first block may lack an opening tag (the prefill case). Every later block requires an explicit <think> / <thinking> |
| 4 | Unclosed-tail handling used rfind, hijacking prose containing a literal <think> | Only triggered when the body starts with an open tag |
| 5 | continue_final_message trimmed the prefill, destroying trailing spaces/newlines | Prefill content is emitted byte-exact |
| 6 | continue_final_message=true on a non-assistant last message produced a prompt with no assistant header | Only honoured when the last message is assistant; otherwise normal generation prompt |
| 7 | Error detection: 'import ' in head suppressed strong errors, so pytest failures (> import foo / E ModuleNotFoundError) scored as success | Split into file_like (suppresses all) / code_like (suppresses weak only) / trace_like (re-enables strong). Added message-level is_error and isError |
| 8 | Explicit reasoning_content + identical inline think block were concatenated | Substring dedupe |
| 9 | Dead `count(' | |
| < | im_end | > |
**Grouped tool responses with a live-streak warning:**
```text
<|im_start|>user
<tool_response>
{"ok": true, "bytes": 4210}
</tool_response>
<tool_response>
Error: no such file or directory
⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with corrected arguments.
</tool_response>
<|im_end|>
Creative continuation (task_mode="creative", continue_final_message=True): the prompt ends mid-sentence, byte-exact, with no <|im_end|> — the model resumes at the cut point.
The template ships with a self-contained harness. Run it against your renderer before deploying — it covers both v4.0 regressions and the history-corruption class of bugs.
# pip install jinja2
import json
from jinja2.sandbox import ImmutableSandboxedEnvironment
env = ImmutableSandboxedEnvironment(trim_blocks=True, lstrip_blocks=True)
env.filters["tojson"] = lambda x, **k: json.dumps(x, ensure_ascii=False)
env.globals["raise_exception"] = lambda m: (_ for _ in ()).throw(ValueError(m))
T = env.from_string(open("chat_template.jinja", encoding="utf-8").read())
R = lambda m, **k: T.render(messages=m, **k)
tools = [{"type": "function", "function": {"name": "read_file", "parameters": {}}}]
# 1. no-think prefill is byte-exact
o = R([{"role": "user", "content": "hi"}], enable_thinking=False, add_generation_prompt=True)
assert o.endswith("<|im_start|>assistant\n<think>\n\n</think>\n\n"), repr(o[-40:])
# 2. IMPORTANT names the tag
assert "inside the <think></think> block" in R([{"role": "user", "content": "hi"}], tools=tools)
# 3. literal </think> in prose must NOT be eaten (pruning is on)
prose = "Close reasoning with </think> and open with <think> like this."
o = R([{"role": "user", "content": "q"},
{"role": "assistant", "content": "Answer: " + prose},
{"role": "user", "content": "next"}], add_generation_prompt=True)
assert "Answer: " in o, o
# 4. assistant with NO content key + string args (auto format)
o = R([{"role": "user", "content": "q"},
{"role": "assistant", "tool_calls": [{"function": {"name": "read_file", "arguments": '{"path":"a.py"}'}}]},
{"role": "tool", "content": "ok"}], tools=tools, add_generation_prompt=True)
assert '"arguments": {"path":"a.py"}' in o
# 5. parallel calls + grouped tool responses stay balanced
o = R([{"role": "user", "content": "q"},
{"role": "assistant", "tool_calls": [{"function": {"name": "f", "arguments": {}}},
{"function": {"name": "g", "arguments": {}}}]},
{"role": "tool", "content": "a"}, {"role": "tool", "content": "b"}],
tools=tools, tool_call_format="qwen3_coder", add_generation_prompt=True)
assert o.count("Tool … does not exists.Tool read_file does not exists.Tool read_file does not exists.
---
license: apache-2.0
language:
- en
- multilingual
pipeline_tag: text-generation
library_name: transformers
tags:
- chat-template
- jinja
- qwen
- qwen2.5
- qwen3
- qwq
- function-calling
- tool-use
- agentic
- agent
- reasoning
- chain-of-thought
- cot
- creative-writing
- vllm
- sglang
- llama.cpp
- transformers
---
# Improved Chat Template for Qwen 3.x — v4.1
A single, production-hardened **Jinja2 chat template** for Qwen 2.5 / Qwen 3.x / QwQ and hybrid reasoning + tool-calling merges (KAT-Coder, Qwopus, AgentWorld, Ornith lineage).
**One template, three postures:**
| Posture | When it activates | What it does |
|---|---|---|
| **Agent** | tools are present | dual-syntax tool calling, multi-turn tool loops, failure circuit breaker |
| **Coding** | tools are present (default agent posture) | adds engineering discipline: read-before-edit, parallel reads, verify-before-success |
| **Creative** | `task_mode="creative"` or `<\|mode_creative\|>` | drops all agentic scaffolding, adds craft guidance, preserves formatting byte-exactly |
Everything is opt-out. With no tools and no knobs set, the rendered prompt is as clean as a stock Qwen template.
- **Template file:** `chat_template.jinja`
- **Internal version string:** `qwen-hybrid-agentic-v4.1`
- **Previous version:** `chat_template_old.jinja` (v3, kept for rollback)
---
## What's new in v4.1
v4.1 is a full rewrite followed by a line-by-line audit. The **on-the-wire format is unchanged** from v3 — same tags, same tool-call serialization, same tool-response grouping — so it is a drop-in replacement.
### Correctness fixes
| Area | Problem in v3 | v4.1 |
|---|---|---|
| **Reasoning extraction** | Any literal `</think>` in prose was treated as a reasoning boundary. Combined with pruning, this could **silently delete assistant answer text** from history. | Only the *first* block may lack an opening tag (the prefill case). Every later block requires an explicit `<think>`/`<thinking>`. |
| **Truncated turns** | `rfind` on an unclosed `<think>` could hijack the tail of a legitimate answer. | Unclosed-tail extraction only when the body *starts* with the open tag. |
| **Interleaved thinking** | Only the first think block was extracted; later ones leaked raw into content. | All blocks extracted in order, preserving any prose around them. |
| **String tool arguments** | OpenAI wire format (`arguments` as a JSON *string*) was dumped as an unparseable blob inside `<function=…>`. | `tool_call_format="auto"` renders those turns in canonical JSON syntax — lossless, no JSON parser required. |
| **`tools` as a string** | Iterated character-by-character, emitting garbage schemas. | Strings are rejected by the type guard. |
| **Duplicate reasoning** | Explicit `reasoning_content` + an inline think block were concatenated. | Substring dedupe. |
| **Error detection** | `'import ' in head` suppressed *strong* errors, so pytest failures (`> import foo` / `E ModuleNotFoundError`) scored as success. | Split into `file_like` / `code_like` / `trace_like`; structured `is_error` / `isError` honoured first. |
| **Stale warnings** | Every historical failure re-emitted a warning on every turn, forever. | Only the **live, unresolved** trailing failure streak is escalated. |
| **Truncation** | Head-only clipping cut off the end of logs — exactly where errors live. | Head **+ tail** with an explicit omission marker; JSON payloads flagged as unparseable. |
| **`enable_thinking=false`** | Silently overridden by `reasoning_effort="high"`. | Explicit disable wins. |
| **Directive precedence** | First-match `elif` chains; a message with two directives honoured only one. | Last occurrence wins, per independent family (think / mode / terse). |
| **Missing shapes** | `role="function"`, single-mapping `tool_calls`, mapping `content`, nested content blocks. | All handled. |
| **Creative continuation** | Trailing whitespace of a prefill was trimmed, breaking seamless continuation. | `continue_final_message` emits the prefill byte-exactly. |
### Behaviour changes (documented, reversible)
| Knob | v3 default | v4.1 default | Why |
|---|---|---|---|
| `tool_call_format` | `qwen3_coder` | `auto` | keeps history canonical and parseable when the harness round-trips string arguments |
| `preserve_thinking` | `true` (keep everything) | `auto` (current segment) | largest context saving in long agent loops |
Both revert with a single kwarg — see [Restoring v3 behaviour](#restoring-v3-behaviour).
### Removed
- `add_vision_id` and all vision-pad macros. This template targets **pure-text CausalLM** merges with the vision tower removed; non-text blocks are rejected or placeholdered via `media_policy`.
---
## Quick start
```python
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
tok.chat_template = open("chat_template.jinja", encoding="utf-8").read()
prompt = tok.apply_chat_template(
[{"role": "user", "content": "Refactor this module and run the tests."}],
tools=tools,
tokenize=False,
add_generation_prompt=True,
reasoning_effort="high",
max_tool_response_chars=12000,
)
Common pitfall: in 🤗 Transformers, template knobs are passed as direct keyword arguments. Do not nest them under
chat_template_kwargs={...}— that key is a vLLM / OpenAI-request concept and will be ignored byapply_chat_template.
All parameters are optional. Pass them as template kwargs (Transformers) or via chat_template_kwargs (vLLM / SGLang request bodies).
| Parameter | Type | Default | Description |
|---|---|---|---|
enable_thinking | bool/str | true | Master switch for the reasoning channel. Accepts true/false, "true"/"false", 1/0, "on"/"off", "yes"/"no" (case- and whitespace-tolerant). |
reasoning_effort | str | "medium" | none/off · minimal/low · medium · high/xhigh/max/ultracode/extreme/deep/long. medium injects nothing (clean prefix). |
preserve_thinking | str/bool | "auto" | auto = keep reasoning from the last real user query onward · true = keep all · false = drop all. Aliases: preserve_reasoning, "last", "segment". |
prefill_think | bool | true | Prefill <think>\n when thinking is on. Set false to let the model open the block itself. |
auto_disable_thinking_with_tools | bool | false | Force thinking off whenever tools are present (low-latency routing). |
| Parameter | Type | Default | Description |
|---|---|---|---|
tool_call_format | str | "auto" | auto · qwen3_coder (XML, xml accepted as alias) · json. See auto selection. |
suppress_tool_instructions | bool | auto | Suppress the injected tool block. Auto-detects runtime protocols ([TOOL_REQUEST], [AVAILABLE_TOOLS], <tool_protocol>, <available_tools>, <tools>) already in your system prompt. |
tool_response_labels | bool | false | Emit name= / id= attributes on <tool_response> for parallel-call attribution. |
failure_warnings | bool | true | Enable the anti-loop circuit breaker. |
failure_warning_scope | str | "active" | active = warn only on the live trailing failure streak · all = v3 behaviour (warn on every historical failure). |
max_tool_arg_chars | int | 0 (off) | Clip tool-call argument values in history. |
max_tool_response_chars | int | 0 (off) | Clip tool outputs in history. Recommended: 8000–16000 for coding agents. |
truncation_mode | str | "head_tail" | head_tail keeps 60% head + 40% tail with an omission marker (falls back to head-only below 200 chars) · head = v3 behaviour. |
| Parameter | Type | Default | Description |
|---|---|---|---|
task_mode | str | "auto" | auto → agent if tools else chat. Explicit: agent/coding/code/agentic · creative/writing/story/prose · chat/plain. |
agent_guidance | bool | true | Inject the Engineering discipline block when tools are active. |
creative_guidance | bool | true | Inject the Craft block in creative mode. |
terse | bool | false | Conciseness rules. Forced off in creative mode. |
| Parameter | Type | Default | Description |
|---|---|---|---|
media_policy | str | "error" | error = raise on image/audio/video blocks · placeholder = inline note, loop keeps running · drop = silently omit. |
continue_final_message | bool | false | Leave the last assistant turn open and emit its content verbatim (no trim, no <|im_end|>). Only honoured when the last message is assistant. |
add_generation_prompt | bool | false | Append the assistant header + thinking prefill. |
Directives are stripped from the rendered text — no token leakage. Within each family, the last occurrence wins; families are independent, so <|mode_creative|> and <|think_xhigh|> can coexist in one message.
| Family | Tokens |
|---|---|
| Thinking off | <|think_off|>, <|no_think|> |
| Thinking on | <|think_on|>, <|think|> |
| Low effort | <|think_low|>, <|think_minimal|> |
| Medium effort | <|think_medium|> |
| Max effort | <|think_xhigh|>, <|think_high|>, <|think_max|>, <|think_ultracode|>, <|think_extreme|>, <|think_deep|>, <|think_long|> |
| Posture | <|mode_creative|>, <|creative|>, <|mode_agent|>, <|mode_code|>, <|coding|>, <|mode_chat|>, <|chat|> |
| Terseness | <|terse_on|>, <|terse_off|> |
| Qwen soft switches | /think, /no_think — honoured only at the very start or end of a message, so URLs and prose are never damaged |
Directives are read from system, developer, and user messages only.
tool_call_format="auto" resolves in this order:
<function= → XML, <tool_call>{"name" → JSON).<parameter=> blocks without a JSON parser, which portable Jinja runtimes do not provide.qwen3_coder XML.This keeps history canonical and re-parseable instead of silently losing arguments. Cache note: if your harness starts passing string arguments mid-session, the system block switches syntax once and then stays stable. Pin tool_call_format explicitly if you need a byte-stable prefix.
reasoning_content → thinking → reasoning (string or {"text": …}), then deduped against inline blocks.preserve_thinking="auto" keeps reasoning from the last real user message onward (user messages that are pure <tool_response> wrappers don't count). With no real user message at all, only the final turn keeps its reasoning.Consecutive tool failures are counted across tool, legacy function, and user-wrapped <tool_response> messages, and reset on any success or real user turn.
| Streak | Injected signal |
|---|---|
| 1 | Diagnose the failure, retry with corrected arguments |
| 2 | Change approach — repeating is unlikely to work |
| 3–4 | Critical loop — pivot tool or report the blocker |
| ≥5 | Hard stop — take a fundamentally different route or report to the user |
Detection prefers structured signals (is_error, isError, "success": false, "status": "error", non-zero exit codes) over text heuristics, recognises explicit success payloads, and distinguishes diffs/file listings from real tracebacks.
Nothing dynamic (no counters, no timestamps) is injected into the system block. Guidance blocks are placed before your system prompt so your instructions always take precedence. Failure warnings live at the tail, inside the tool response.
Thinking disabled, generation prompt:
<|im_start|>assistant
<think>
</think>
XML tool call (parallel calls stay separate, fully closed blocks):
<|im_start|>assistant
<think>
Need both files before editing.
</think>
<|im_end|>
Consecutive tool messages merge into one user turn:
<|im_start|>user
<tool_response>
…first output…
</tool_response>
<tool_response>
Traceback (most recent call last): …
⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with corrected arguments.
</tool_response>
<|im_end|>
vllm serve Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder
Use --tool-call-parser hermes when running tool_call_format="json". Parser and reasoning-parser names vary by vLLM version — check yours.
Per-request overrides:
{
"model": "Qwen/Qwen3-32B",
"messages": [{"role": "user", "content": "Draft the opening scene."}],
"chat_template_kwargs": {
"task_mode": "creative",
"reasoning_effort": "high",
"preserve_thinking": "auto"
}
}
python3 -m sglang.launch_server \
--model-path Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--port 30000
llama-server -m model.gguf --jinja --chat-template-file ./chat_template.jinja
model_list:
- model_name: qwen-agent
litellm_params:
model: vllm/Qwen/Qwen3-32B
api_base: http://localhost:8000/v1
Serve the template from the backend (vLLM/SGLang) rather than the proxy so tool parsing and reasoning parsing stay aligned.
The template deliberately restricts itself to constructs available across 🤗 Transformers, vLLM, SGLang and minja (llama.cpp):
do extension, no break/continue, no while+ concatenation, no .append() (blocked by ImmutableSandboxedEnvironment)fromjson filter, no extended slices ([::-1])namespace(); all loops are boundedraise_exception and tojson are required from the host environmentChatML framing is standard Qwen: <|im_start|> / <|im_end|>, no BOS injection.
<think>…</think> pair will still be interpreted as a reasoning block. This is inherent to the format; v4.1 removes the dangerous unpaired cases.{"type": "tool_use", …} inside content) are JSON-dumped rather than converted to tool-call markup. Structured message.tool_calls with .arguments / .parameters / .input works fully.A single, production-hardened Jinja2 chat template for Qwen 2.5 / Qwen 3.x / QwQ and hybrid reasoning merges (KAT-Coder · Qwopus · AgentWorld · Ornith).
One template, three postures:
| Posture | When it activates | What it does |
|---|---|---|
| 🛠️ Agent | tools are supplied | Dual-syntax tool calling, multi-turn tool loops, failure circuit breaker |
| 💻 Coding | tools active (default) | Agent + engineering discipline: read-before-edit, parallel reads, verify-before-success |
| ✍️ Creative | task_mode="creative" or <|mode_creative|> | Long-form prose, zero agentic scaffolding, byte-exact formatting & continuation |
No posture is forced. With no tools and no mode set, the template renders a clean, neutral ChatML prompt.
qwen-hybrid-agentic-v4.1chat_template.jinjachat_template_old.jinja (v3, kept for rollback)v4.1 is an audit release. It fixes two regressions introduced during the v4 rewrite plus a set of latent defects inherited from v3, without changing the on-the-wire format.
| Severity | Issue | Resolution |
|---|---|---|
| High | No-think prefill emitted <think>\n\n</think>\n (missing blank line vs. canonical Qwen3) | Restored byte-exact <think>\n\n</think>\n\n |
| High | The XML <IMPORTANT> rule lost its tag literal ("…inside the block") | Restored <think></think> |
| High | Any literal </think> in prose was treated as a reasoning boundary — the preceding answer text was swallowed into "reasoning" and then silently deleted by history pruning | Only the first block may be open-tag-less (the prefill case); every later block requires an explicit <think> / <thinking> |
| Medium | Unclosed-tail extraction used rfind, hijacking prose that merely mentions <think> | Only triggers when the body starts with an open tag |
| Medium | continue_final_message trimmed the prefill, destroying trailing spaces/newlines | Prefill content is now emitted byte-exact |
| Medium | continue_final_message=true on a non-assistant last message produced a prompt with no assistant header | Only honoured when the last message is assistant |
| Medium | 'import ' in head suppressed strong errors, so pytest failures (> import foo / E ModuleNotFoundError) scored as success | Split into file_like / code_like / trace_like; added message-level is_error / isError |
| Low | reasoning_content and an identical inline think block were concatenated | Substring dedupe |
| Low | Dead `count(' |
**JSON tool call**
```text
Parallel tool responses — consecutive tool messages are merged into a single user turn:
<|im_start|>user
<tool_response>
…result A…
</tool_response>
<tool_response>
…result B…
</tool_response>
<|im_end|>
vllm serve Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder
Use --tool-call-parser hermes when running tool_call_format="json". Parser names vary by vLLM version — check yours. Pair with the matching reasoning parser if you want reasoning_content split out server-side.
Per-request kwargs:
{
"model": "Qwen/Qwen3-32B",
"messages": [{"role": "user", "content": "Draft the opening scene."}],
"chat_template_kwargs": {
"task_mode": "creative",
"reasoning_effort": "xhigh"
}
}
python3 -m sglang.launch_server \
--model-path Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--port 30000
llama-server -m model.gguf --jinja --chat-template-file ./chat_template.jinja
model_list:
- model_name: qwen-agent
litellm_params:
model: vllm/Qwen/Qwen3-32B
api_base: http://localhost:8000/v1
Point the backing server at chat_template.jinja; forward chat_template_kwargs from the client.
{
"max_tool_response_chars": 12000,
"truncation_mode": "head_tail",
"preserve_thinking": "auto",
"media_policy": "placeholder",
"reasoning_effort": "medium"
}
{
"task_mode": "creative",
"reasoning_effort": "xhigh",
"preserve_thinking": "auto",
"continue_final_message": true
}
The template deliberately avoids anything a sandboxed renderer might not implement:
do, no break/continue, no while+ concatenation, no .append (blocked by ImmutableSandboxedEnvironment)fromjson, no extended slices ([::-1])namespace, string ops, tojson, raise_exception, and core filtersTested targets: 🤗 Transformers, vLLM, SGLang, llama.cpp/minja. All v3 knob names remain valid.
Removed: add_vision_id and all vision-pad macros — this template targets pure-text CausalLMs.
<think> / </think> can still be interpreted as a boundary. v4.1 narrows this to the first block only, but it is inherent to the format.{"type": "tool_use", …} inside content) are JSON-dumped rather than converted to `
Tool write_file does not exists.license: apache-2.0 language:
A single, production-hardened Jinja2 chat template for Qwen 2.5 / Qwen 3.x / QwQ and hybrid reasoning‑tool merges (KAT‑Coder · Qwopus · AgentWorld · Ornith lineage).
One template, three postures: agentic tool calling, engineering‑disciplined coding, and long‑form creative writing — without the scaffolding of one mode bleeding into another.
chat_template.jinjaqwen-hybrid-agentic-v4.1<|im_start|> / <|im_end|>, Qwen XML and JSON tool syntax, grouped <tool_response> blocks). This release fixes correctness bugs; it does not retrain your model's expectations.v4.1 is an audit release. It repairs two regressions introduced in v4.0 and several latent defects inherited from v3 — including one that could silently delete assistant text from history. See What changed.
| File | What it is |
|---|---|
chat_template.jinja | v4.1 — use this. Unified agent / coding / creative template. |
chat_template_old.jinja | v3 agentic template (previous chat_template.jinja), kept for rollback / diffing. |
previous-version-chat_template.jinja | Older historical revision. |
aesthetic_chat_template.jinja | Standalone creative‑writing variant, kept for reference. v4.1's task_mode="creative" covers this inside the unified template. |
tokenizer_config.json, config.json, generation_config.json | Convenience configs. The template is standalone — you do not need these to use it. |
| # | Severity | Issue | Fix |
|---|---|---|---|
| 1 | High | v4.0 emitted <think>\n\n</think>\n; canonical Qwen3 / v3 emit <think>\n\n</think>\n\n. | Restored byte‑exact. |
| 2 | High | The XML <IMPORTANT> rule lost its tag literal ("…inside the block."). | Restored <think></think>. |
| 3 | High | Any literal </think> in prose was treated as a reasoning boundary. The preceding answer text was reclassified as "reasoning" — and with pruning enabled, silently deleted from history. | Only the first block may be open‑tag‑less (the prefill case). Every later block requires an explicit <think> / <thinking>. |
| 4 | Medium | Unclosed‑tail extraction used rfind, so a literal <think> mid‑prose hijacked the tail. | Only fires when the body starts with the open tag. |
| 5 | Medium | continue_final_message trimmed the prefill, destroying trailing spaces / newlines and breaking seamless prose continuation. | Prefill content is emitted byte‑exact. |
| 6 | Medium | continue_final_message=true with a non‑assistant last message produced a prompt with no assistant header. | Only honoured when the last message is assistant; otherwise normal generation prompt. |
| 7 | Medium | Error detection: 'import ' in head suppressed strong errors, so pytest output (> import foo / E ModuleNotFoundError) scored as success. | Split into file_like (suppresses all), code_like (suppresses weak only), trace_like (re‑enables strong). Added message‑level is_error / isError. |
| 8 | Low | Explicit reasoning_content and an identical inline think block were concatenated → duplicated reasoning. | Substring dedupe. |
| 9 | Low | Dead `count(' |
Tool call, JSON:
Consecutive tool results are merged into **one** user turn:
<|im_start|>user <tool_response> …first result… </tool_response> <tool_response> …second result… ⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with corrected arguments. </tool_response> <|im_end|>
---
## 🛠️ Deployment
### vLLM
```bash
vllm serve Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser qwen3
Use --tool-call-parser hermes when running tool_call_format="json". Parser names vary by vLLM version — check yours.
Per‑request overrides:
{
"model": "Qwen/Qwen3-32B",
"messages": [{"role": "user", "content": "Refactor the parser."}],
"chat_template_kwargs": {
"task_mode": "coding",
"reasoning_effort": "xhigh",
"max_tool_response_chars": 12000
}
}
python3 -m sglang.launch_server \
--model-path Qwen/Qwen3-32B \
--chat-template ./chat_template.jinja \
--port 30000
llama-server -m model.gguf --jinja --chat-template-file ./chat_template.jinja
model_list:
- model_name: qwen-agent
litellm_params:
model: vllm/Qwen/Qwen3-32B
api_base: http://localhost:8000/v1
Point the backing server at chat_template.jinja; pass posture/effort through chat_template_kwargs.
Written to the lowest common denominator of HF ImmutableSandboxedEnvironment, vLLM, SGLang and minja (llama.cpp):
do, break/continue, while, fromjson, list +, .append, or extended slices.namespace() (Jinja loop scoping is respected everywhere).\n escapes rather than raw newlines inside strings.Known limitations
tool_use / tool_result content blocks are JSON‑dumped rather than converted to `Tool … does not exists.Tool edit_file does not exists.Tool … does not exists.license: apache-2.0 language:
A single, production-hardened Jinja2 chat template for Qwen 2.5 / Qwen 3.x / QwQ, hybrid reasoning merges, and tool-calling agents.
One template, three postures — it adapts to what you are actually doing instead of forcing every task through an agent-shaped prompt:
| Posture | When | What it does |
|---|---|---|
| 🛠️ Agent | tools are supplied | Dual-syntax tool calling, multi-turn tool loops, failure circuit breaker |
| 💻 Coding | tools + code work | Agent behaviour plus a compact engineering-discipline block |
| ✍️ Creative | task_mode="creative" | Long-form prose: no agentic scaffolding, no terseness, byte-exact continuation |
Internal version string: qwen-hybrid-agentic-v4.1.
Main file: chat_template.jinja (previous generation preserved as chat_template_old.jinja).
v4.1 is an audit release. The on-the-wire format is unchanged from v3 — same ChatML tags, same tool-call serialization, same tool-response grouping — so it is a drop-in replacement. Everything below is correctness, safety, or control.
| Area | Problem in previous versions | v4.1 |
|---|---|---|
| Reasoning extraction | Any literal </think> in prose was treated as a reasoning boundary. Combined with history pruning this could silently delete assistant answer text. | Only the first block may be open-tag-less (the prefill case). Every later block requires an explicit <think> / <thinking>. |
| Truncated turns | rfind on an unclosed <think> could hijack the tail of a normal answer. | Unclosed-tail extraction only when the body starts with the tag. |
| Interleaved thinking | Only the first think block was extracted; later ones leaked raw tags into content. | All blocks extracted in order, preserving surrounding prose. |
| Duplicate reasoning | reasoning_content and an inline think block with the same text were concatenated. | Substring dedupe. |
| String tool arguments | OpenAI-style arguments (a JSON string) were dumped into <function=…> with no <parameter=> tags — unparseable, and the model saw its own past calls with unreadable args. | tool_call_format="auto" renders those turns in canonical JSON. No hand-rolled JSON parser required. |
tools as a string | A JSON-string tools value was iterated character by character. | Strings excluded from the tool-schema path. |
| Error detection | 'import ' in the head suppressed all errors — pytest output (> import foo / E ModuleNotFoundError) scored as success. | Split into file_like / code_like / trace_like; strong errors survive code-looking output. Message-level is_error / isError honoured first. |
| Stale warnings | Every historical failure re-emitted a warning on every turn, forever. | Escalation only fires on the live, unresolved failure streak (failure_warning_scope="all" restores the old behaviour). |
| Truncation | Head-only clipping cut off the end of logs — exactly where errors live. | Head + tail with an explicit omission marker. |
enable_thinking=false | Silently overridden by reasoning_effort="high". | Explicit disable wins. |
Missing content key | Relied on Undefined being iterable. | none / undefined checked first. |
| Nested content blocks | {"content": [ … ]} was JSON-dumped into the prompt. | Recursively rendered. |
| Legacy shapes | role="function", single-mapping tool_calls, mapping content were dropped. | All handled. |
task_mode — auto / agent / coding / creative / chat (aliases: code, writing, story, prose, plain).continue_final_message — leaves the last assistant turn open and emits its content byte-exact (trailing spaces and newlines preserved) for seamless prose continuation and prefills.media_policy — error (default) / placeholder / drop, so a stray screenshot in an agent harness no longer kills the request.tool_response_labels — optional name= / id= attributes on <tool_response> for parallel-call attribution./think and /no_think, honoured only at the very start or end of a message (no false positives on URLs).| Knob | Old default | New default | Why |
|---|---|---|---|
tool_call_format | qwen3_coder | auto | Keeps history canonical and parseable |
preserve_thinking | true | auto | Prunes reasoning older than the current user turn — the single biggest context saver in agent loops |
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
tok.chat_template = open("chat_template.jinja", encoding="utf-8").read()
prompt = tok.apply_chat_template(
[{"role": "user", "content": "Refactor this module and run the tests."}],
tools=tools,
tokenize=False,
add_generation_prompt=True,
reasoning_effort="high",
max_tool_response_chars=12000,
)
⚠️ Common pitfall: in 🤗 Transformers, template kwargs are passed directly to
apply_chat_template(...). Do not nest them underchat_template_kwargs={...}— that key is a vLLM / OpenAI-request convention, not a Transformers one.
Creative mode, continuing a scene mid-sentence:
prompt = tok.apply_chat_template(
[{"role": "user", "content": "Continue the scene."},
{"role": "assistant", "content": "The rain fell on the "}],
tokenize=False,
task_mode="creative",
continue_final_message=True,
)
# ends exactly with "The rain fell on the " — no <|im_end|>, no trimming
All parameters are optional. Pass them as template kwargs (Transformers) or via chat_template_kwargs (vLLM / SGLang / proxies). String booleans ("true", "false", "1", "on", "yes") and integer strings are normalized — no TypeError behind FastAPI gateways.
| Parameter | Type | Default | Description |
|---|---|---|---|
enable_thinking | bool/str | true | Master switch for the reasoning channel. |
reasoning_effort | str | "medium" | none/off → thinking off · minimal/low → concise · medium → no extra instruction · high/xhigh/max/ultracode/extreme/deep/long → rigorous. |
preserve_thinking | str/bool | "auto" | auto keeps reasoning from the last real user message onward · true keeps everything · false drops all history reasoning. Alias: preserve_reasoning. |
prefill_think | bool | true | Prefill <think>\n when thinking is on. Set false to let the model open the block itself. |
auto_disable_thinking_with_tools | bool | false | Low-latency agent routing: disable reasoning whenever tools are present. |
| Parameter | Type | Default | Description |
|---|---|---|---|
tool_call_format | str | "auto" | auto · qwen3_coder (XML, xml accepted as alias) · json. |
suppress_tool_instructions | bool | auto | Suppress the injected tool block. Auto-detects runtime protocols ([TOOL_REQUEST], [AVAILABLE_TOOLS], <tool_protocol>, <available_tools>, <tools>). |
agent_guidance | bool | true | Engineering-discipline block when tools are active. |
tool_response_labels | bool | false | Add name= / id= to <tool_response> for parallel calls. |
failure_warnings | bool | true | Anti-loop circuit breaker. |
failure_warning_scope | str | "active" | active = only the live failure streak · all = every historical failure. |
max_tool_arg_chars | int | 0 (off) | Clip tool arguments in history. |
max_tool_response_chars | int | 0 (off) | Clip tool outputs. Recommended: 8000–16000 for agents. |
truncation_mode | str | "head_tail" | head_tail keeps 60 % head + 40 % tail · head keeps the prefix only. |
| Parameter | Type | Default | Description |
|---|---|---|---|
task_mode | str | "auto" | auto → agent when tools exist, otherwise chat. Also agent/coding, creative, chat. |
creative_guidance | bool | true | Craft block in creative mode. |
terse | bool | false | Conciseness rules. Forced off in creative mode. |
| Parameter | Type | Default | Description |
|---|---|---|---|
media_policy | str | "error" | Image/audio/video blocks: error raises · placeholder inserts a text note · drop removes silently. |
continue_final_message | bool | false | Leave the last assistant turn open, content emitted verbatim. Only honoured when the last message is assistant. |
add_generation_prompt | bool | false | Append the assistant generation header. |
Drop these anywhere in a system / developer / user message. They are stripped from the rendered prompt — no token leakage. Last occurrence wins, and the three families are independent.
| Family | Tokens |
|---|---|
| Thinking off | `< |
| Thinking on | `< |
| Low effort | `< |
| Medium effort | `< |
| Deep effort | `< |
| Posture | `< |
| Terseness | `< |
| Qwen3 soft switch | /think /no_think — honoured only at the start or end of a message |
auto)tool_call_format always wins.<function=…> → XML, {"name": …} → JSON).<parameter=> blocks without a JSON parser, which portable Jinja runtimes do not provide.qwen3_coder).Cache note: step 3 can flip the system block once, mid-conversation. Pin tool_call_format="qwen3_coder" or "json" if you need a byte-stable prefix.
reasoning_content, thinking, or reasoning (string or {"text": …}), then merged with any inline think blocks — deduplicated.