Qwen3.6-27B-A3B-CoderX-MTP-GGUF

Qwen3.6-27B-A3B-CoderX — GGUF (imatrix, MTP)

GGUF quantizations of CoderX, a long-horizon code prune of Qwen3.6-35B-A3B: the MoE is reduced from 256 experts to 184 (72 dropped per layer, ~35B→27B, still A3B active).

Same expert budget as the sibling Qwen3.6-27B-A3B-Coder, but a different selection — same competence map, different keep-set. Served at top-8 — the base model's native routing, no override needed. No fine-tuning, no distillation.

CoderX is not a strict upgrade over Coder. It is the long-horizon sibling: far stronger on multi-step competitive-programming work, a tie on short multi-language completion, and its non-code profile is not yet measured. Pick by workload.

Recipe

1. Competence map: the 256e teacher is profiled per-expert on a balanced corpus plus targeted LiveCodeBench and MultiPL-E/HumanEval PASS-response classes.
2. Selection: our saliency ranking picks the 184 survivors, with a REAP-style per-layer floor (p=24) protecting the tail — the constraint the plain saliency cut over-prunes.
3. No redistribution: the 72 evicted experts per layer are discarded, not merged, and every retained expert stays bit-identical to the base model's. Expert-folding variants were built and measured against this cut, and rejected — they cost 14pp on LiveCodeBench-v6 and drove the generation-loop rate from 16% to 78%.
4. Top-8 routing — the base default, left alone. The Coder sibling bakes top-10; this model does not, and that was measured rather than assumed (below).

Evaluation (Q6_K + imatrix, llama.cpp b9700, greedy)

▲ = beats both the 256e teacher and the Coder sibling. Every cell is greedy (temperature 0.0 / top_p 1.0 / top_k 0), one serving geometry per bench, pinned in the eval template and read back out of the server log rather than chosen at launch time. All three models ran on the same binary, the same quant recipe and the same basis.

Read the MultiPL-E row carefully. A same-basis repeat of that bench — identical weights, quant, sampler and geometry — moved 1.0 pp on batch-scheduling nondeterminism alone. The 0.33 pp CoderX↔Coder gap sits inside that band and should be read as a tie; the 2.33 pp gap to the teacher sits outside it and is real. Per language (CoderX / Coder / 256e): Rust 0.85 / 0.81 / 0.84 · Java 0.89 / 0.90 / 0.93 · JS 0.92 / 0.96 / 0.96. CoderX takes Rust and gives up JavaScript.

Routing ships top-8, and that was measured: MBPP-full 0.784 / 0.790 at top-8 against 0.732 / 0.730 at top-10, with repeat spreads ≤ 0.3 pp on both arms. MultiPL-E was inside noise either way.

Not measured yet. The full canonical 9-bench suite has not been run on this checkpoint. GPQA, MATH-500, IFEval and ARC are deliberately absent from the table above rather than carried over from a different sampler or a different basis — code-targeted expert cuts in this family are known to cost science and multilingual headroom, so treat the non-code profile as unknown until those cells land. There is also no per-tier quality sweep yet: the quant table below is sizes and imatrix provenance only.

Quantizations

Every K- and I-quant tier is built with an importance matrix. This was verified by reading the quantize.imatrix.* keys back out of each uploaded file rather than trusting the build flag — all 18 quant tiers report calibration_datav5.txt, 510 entries, 128 chunks. Q8_0 carries none by rule. imatrix.dat is archived in this repo, so any tier can be reproduced or re-quantized to a matching characteristic.

On this model family the imatrix is load-bearing at 4-bit — the opposite of Gemma-4, where it degrades K-quants.

F16 (48.87 GiB / 52.47 GB) is also present as the quantization source.

Both units are given because they get confused constantly: HF reports GiB, the ollama page quotes GB. Sizes are the actual file bytes, so the ladder is ordered by size rather than by name — Q3_K_L really is larger than Q3_K_XL on this architecture.

No ContribDynamic CD-* tiers in this repo yet — unlike the Coder sibling. If you want the CD ladder, use Coder for now.

Speculative decoding — the MTP head ships in every tier

The native MTP block (blk.40) is included in every GGUF here, so speculative decoding works without a separate draft model.

On ollama it is already wired: every tag carries PARAMETER draft_num_predict 3, which runs --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-backend-sampling.

n=3 is a deliberate choice, not a default. On an RTX 5080 (Blackwell): 189.65 tok/s baseline → 251.94 at n=3 (+33%), falling to 145.37 at n=8 — worse than not speculating at all. On a 3090 the same grid peaks at n=8 instead. The optimum is hardware-dependent; n=3 is near-peak on both, so it is the shipped default. Raise it yourself if you are on Ampere and measuring. Acceptance at n=3: mean accepted length 3.16, per-position (0.841, 0.730, 0.587).

Vision

The vision-<tier> tags on ollama attach the released Qwen3.6-27B-A3B-Coder mmproj (clip projector, projection_dim 2048, matching this model's embedding_length 2048).

Note that the bf16 checkpoint this repo was quantized from is text-only — the base model's vision tower is not carried by that checkpoint (the merge-engine save path emits trunk modules only), so Qwen3.6-27B-A3B-CoderX is Qwen3_5MoeForCausalLM, not the Coder sibling's Qwen3_5MoeForConditionalGeneration. Multimodal use goes through the mmproj at the GGUF layer, which is unaffected.

Serving

llama-server -m Qwen3.6-27B-A3B-CoderX-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

CoderX thinks long. On LiveCodeBench its median completion is ~15.8k tokens against ~2.2k for the published Coder, and 60 of 77 completions trip a repetition heuristic — but 42 of those 60 still pass. The length is where the LCB win comes from, not a defect. If your work is LCB-shaped, give it context headroom rather than clamping it.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the whole context window before it answers. llama.cpp can bound the thinking block with a sampler and — the part that actually matters — tell the model why the block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the per-request overrides.

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking is handed back — message.reasoning_content versus left inline in message.content — and never whether the budget is enforced: the delimiters the sampler counts are set by the chat template regardless, so the cap binds under auto, deepseek and none alike. Leave it at the default so the model's own tool-call and channel handling stays in play, and pin deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget runs out, wherever the model happens to be. When that lands mid-thought the model frequently does not register that it was interrupted: it carries on reasoning, now inside the visible answer. The stop phrase is what prevents that — it gives the model a reason to be finishing.

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message "is not explicitly trained but emerges naturally", so it is worth trying both on your own workload. Leading and trailing newlines matter — they keep the phrase off whatever half-finished line the cut landed on.

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard questions, temperature 0.6, fixed seed, answer characters with wall time in brackets. Every run answered all three correctly, and thinking length is unchanged by the message in every row:

The 4096 row is the failure this exists for: the cap lands mid-thought and the reasoning simply continues in the answer, ten times longer and 2.2x the wall time, for the same three correct answers. Both phrases remove it.

The server accepts both as request fields, overriding the command line:

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have to be supplied with the budget:

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even when empty: llama.cpp builds the sequence it forces from message + end_tag inside that field's handler, so omitting it leaves the budget with nothing to force — the sampler logs as though the cap fired while the thinking block stays open.

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• Qwen recommend keeping a thinking budget above 1024 tokens; below that the cap tends to land before the model has committed to an approach.
• The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Formats

• bf16 safetensors: Qwen3.6-27B-A3B-CoderX (text-only, MTP included).
• Ollama: mannix/qwen3.6-27b-a3b-coderx — 19 text tags plus a vision-<tier> for each, MTP baked in.

Built with omnimergekit.

Notes

• Top-8 is the base default and is what this model was selected and evaluated at.
• Same tokenizer and chat template as the base.
• Research checkpoint. The non-code profile is unmeasured; see the evaluation section.

Apache-2.0.

Tool-calling benchmark — tool-eval-bench hardmode (88 scenarios, 176 pts)

Benchmarked file: Qwen3.6-27B-A3B-CoderX-Q4_K_M.gguf (this repo), served on llama.cpp with MTP speculative decoding enabled. The score below belongs to THIS quant — other tiers in this repo were not run.

A3B-CoderX scores 137.4 ±4.9 of 176, seventh of ten — and +14.2 pts over A3B-Coder (123.2), the largest gap between any two models in this cohort's lower half. The CIs do not overlap.

The gain is concentrated exactly where Coder gives way: Hard Mode 24.6/38 (64.7%) vs 16.8/38 (44.2%), Autonomous Planning 3.8/6 (63.3%) vs 2.4/6 (40.0%), and Instruction Following 9.6/10 (96.0%) vs 8.0/10. Perfect on Tool Selection and Parameter Precision; Localization 5.6/6 (93.3%), Toolset Scale 7.6/8 (95.0%).

Weakest at Safety & Boundaries 17.6/26 (67.7%) and Context & State 15.0/20 (75.0%). 17 safety-critical failures across five seeds — TC-31, TC-34 and TC-60 on every seed, plus TC-35 (Contradictory Parameters) on two. That is the highest count in the cohort; if untrusted tool output is in scope for your deployment, read the safety notes below.

CoderX hits the TC-62 scorer crash on 3 of 5 seeds, so this figure is mildly depressed relative to a corrected harness.

Full cohort

* one seed (s42) is graded on 174 pts, not 176 — see that model's card.

<details> <summary><b>Basis — read before comparing these numbers to anything</b></summary>

• Scorer: tool-eval-bench v2.6.0 (the pip/uv-installed package, verified via tool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimed cf54b4b (v2.6.0-45); that is wrong and has been corrected — no cell ever ran it. All 50 cells ran the same v2.6.0, so the cohort is internally consistent.
• v2.6.0 carries a known scorer crash on TC-62. email_calls[-1] raises IndexError when a model sent no valid CFO email; the orchestrator catches it and returns FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38 scored cells, 2 pts each, and it is not neutral — it concentrates on the weakest models. Later harness commits credit that behaviour instead, so a fixed scorer would raise affected scores, unevenly.
• 5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout, thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).
• Served on llama.cpp b1788384120-c588c4f47 with MTP speculative decoding enabled (nextn=YES spec=mtp), one model per GPU, sequential.
• Quant tiers are not uniform across the cohort (Q4_K_M for the Omnimerge/A3B rows, IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps therefore carry a quantisation component and are not purely architectural.
• Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those were run at 256k context and are a different basis despite the shared scorer version.

</details>