gemma-4-A4B-98e-v6-coder-it-GGUF

gemma-4-A4B-98e-v6-coder-it-GGUF

✅ Chat template — agentic loop fix (2026-06-10)

The chat template inherited from Gemma 4 upstream re-injected each prior tool-call turn's reasoning_content as a <|channel>thought block (for assistant tool-call steps after the last user message). In deep multi-turn agentic / tool-calling use (OpenCode, llama.cpp --jinja, …) this fed the model an accumulating echo of its own reasoning and could trigger verbatim repetition loops. Single-turn chat is unaffected.

Validated on Q6_K: agentic loop rate 33% → 0%, with HumanEval+ 92.07% and MultiPL-E-100 0.66 unchanged (the fix is single-turn-inert). A drop-in chat_template.fixed.jinja and a 1-minute template_loop_unittest.py are provided for audit. The bug is upstream-wide across the Gemma 4 family + third-party requants — see the writeup.

As of 2026-06-10 the corrected template is embedded in every GGUF tier in this repo.

The anti-repetition sampler flags (--min-p 0.05 --repeat-penalty 1.1) noted under How to Use are no longer required to prevent this agentic loop; they remain safe as general sampling. --reasoning-budget 8192 is still required for Gemma 4 thinking.

GGUF quantizations of ManniX-ITA/gemma-4-A4B-98e-v6-coder-it — the LCB-targeted code prune of Gemma 4 26B-A4B (98 experts/layer, ~20.8B). All quants use imatrix with calibration data v5. The recipe + full Q6_K 9-bench scoreboard are in the embedded model card below.

Available Quantizations

Per-tier HumanEval+ (164q, chat-mode, pass@1, greedy) from the llama.cpp sweep (omk_eval humanevalplus_full, canonical summary.json .score), 2-GPU shard on pod 37588132. Q6_K is the 9-bench reference run; F16 is the unquantized baseline. CD-* tiers are ContribDynamic per-layer quants (expert-contribution-aware mixed precision). Sizes are decimal GB (÷10⁹), bpw computed against the cohort 20.8B param count to match the v5-coder card.

imatrix policy. The plain K-quant ladder was rebuilt with imatrix (calibration data v5) on 2026-05-25, which lifted the low-bit K-quants the most (e.g. Q3_K_M 85.98 → 92.68, Q3_K_S 82.93 → 87.80, Q3_K_L/XL 88.41 → 92.07). Q4_K_M is the one exception — built without imatrix, because imatrix measurably lowered its HE+ (90.85 imatrix vs 92.07 plain). IQ-tiers always carry imatrix; Q8_0/Q4_0/Q4_1 never do.

Recommended: IQ4_XS (11.01 GB / 92.07%) is the safe 4-bit default. The imatrix rebuild opened a 92.68% cluster just below it — Q3_K_M (10.51 GB), Q4_K_S (12.21 GB), Q4_K_L (13.42 GB), Q5_K_M (15.07 GB) — so Q3_K_M is now the smallest tier at the top of the HE+ band (the ≤0.61pp steps here are within HE+'s 1-question granularity, so treat the 92.07/92.68 cluster as a tie and pick by disk). v6-coder holds ≥92.07% down to IQ3_XS (9.22 GB / 3.54 bpw). Max fidelity: Q6_K / Q8_0 (93.29%). Avoid IQ2_XS (70.73% — 2-bit cliff; use IQ2_S 89.02% at 7.83 GB instead).

How to Use

With llama.cpp (the canonical eval recipe):

llama-server -m gemma-4-A4B-98e-v6-coder-it-IQ4_XS.gguf \
    -c 32768 -ngl 99 --reasoning-budget 12288 \
    --temp 0.6 --min-p 0.05 --repeat-penalty 1.1

--reasoning-budget is required for Gemma 4 thinking (without it the model emits malformed channel tokens). --temp 0.6 --min-p 0.05 --repeat-penalty 1.1 is anti-repetition sampling: llama.cpp default --repeat-penalty 1.0 lets the thinking channel ruminate (verbatim repetition) on rare ambiguous prompts until it fills the budget; min-p + repeat-penalty eliminates it, and greedy (--temp 0) makes it worse. This is a sampler setting, not a quant-tier defect.

See Reasoning budget and thinking stop phrase below for the budget, the wrap-up phrase that stops reasoning leaking into the answer, and why --reasoning-format plays no part in it.

With ollama:

ollama pull mannix/gemma4-98e-v6-coder:Q4_K_M   # any tier tag; :latest = Q4_K_M

All ollama tags ship these sampler defaults baked in (temperature 0.6, min_p 0.05, repeat_penalty 1.1, num_ctx 8192); raise num_ctx for very long tasks (/set parameter num_ctx 16384).

Reasoning budget and thinking stop phrase (llama.cpp)

On a hard prompt this model will reason until it has consumed the whole context window and then answer with nothing at all. llama.cpp can bound the thinking block with a sampler, and — the part that actually matters — tell the model why the block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the per-request overrides.

Serve with a bounded thinking block

llama-server -m gemma-4-A4B-98e-v6-coder-it-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 1.0 --top-k 64 --top-p 0.95 --min-p 0.05 \
    --repeat-penalty 1.02 --repeat-last-n 2048

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking is handed back — message.reasoning_content versus left inline in message.content — and never whether the budget is enforced: the delimiters the sampler counts are set by the chat template regardless, so the cap binds under auto, deepseek and none alike. The default auto already extracts reasoning and is behaviourally identical to deepseek (they differ only in name; the sole branch in the parser is != none). Leave it at the default so the model's own tool-call and channel handling stays in play, and pin deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget runs out, wherever the model happens to be. When that lands mid-thought the model frequently does not register that it was interrupted: it carries on reasoning, now inside the visible answer. The stop phrase is what prevents that — it gives the model a reason to be finishing.

Two wordings that work

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message "is not explicitly trained but emerges naturally", so it is worth trying both on your own workload. Leading and trailing newlines matter — they keep the phrase off whatever half-finished line the cut landed on.

What it measures out to

Measured on the v7-coder IQ4_NL build of this family, served by the same llama.cpp sampler. Three hard questions, temperature 0.6, fixed seed, answer characters with wall time in brackets. Every run answered all three correctly, and thinking length is unchanged by the message in every row:

AIME 2024, all 30 problems, budget 4096, -c 32768, vendor sampling:

The phrase halves wall time and all but removes the runaway answers — single rows go from 82,067 characters of answer to 1,655. The accuracy differences are inside the noise at n = 30 (paired: qwen −4 net, voice −1 net, exact binomial p ≈ 0.22 and ≈ 1.0), and the terse "Final Answer:" suffix from the s1 paper (arXiv:2501.19393) is not reproducing the accuracy collapse reported there at this budget.

Per request, instead of per server

The server accepts both as request fields, overriding the command line:

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have to be supplied with the budget:

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<|channel>",
  "reasoning_budget_end_tag": "<channel|>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even when empty: llama.cpp builds the sequence it forces from message + end_tag inside that field's handler, so omitting it leaves the budget with nothing to force — the sampler logs as though the cap fired while the thinking block stays open.

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Comparison vs the 14–22B coder field

For sense-of-scale on the Q6_K headline (HE 98.78 / HE+ 93.29 / LCB-medium-55 96.36). Peer numbers are official model-card / paper / blog (linked); cross-lab numbers are noisy by ±2–5pp (different prompts, sampling, chat-vs-base framings). v6-coder is run greedy (T=0).

Coder-specialized 14–22B:

Generalist 14–22B (notable code/reasoning scores):

Where v6-coder sits:

• HE / HE+: top of the band — 98.78 / 93.29, ~+9 / +6pp above Qwen2.5-Coder-14B's 89.6 / 87.2 (the published field leader). Essentially tied with v5-coder (−0.61 HE, identical HE+) while gaining +10.91pp on LCB.
• LCB: v6-coder's 96.36 is LCB-medium-55 on v4 problems. The published Qwen2.5-Coder / DS-Coder-V2-Lite numbers above (23.4 / 24.3) are full LCB on pre-v4 windows — a different, non-comparable subset. The apples-to-apples number now exists: run on the same LCB-medium-55 v4 split and the same llama.cpp stack, Qwen2.5-Coder-14B scores 18.18% vs v6-coder's 96.36%. Read it as a same-rig measurement, not a definitive ranking — the v4 split is only 55 problems, v6-coder was specifically LCB-v4-targeted (that is the entire recipe), and Qwen takes the same llama-server chat-template penalty seen in the same-stack HE+ sweep below. Don't read the raw +78pp as general coding superiority.
• MATH-500 91.00 / GSM8K 91.00 / AIME 63.33: top of the band for math-on-text reasoning. Phi-4's 80.4 MATH is the closest generalist; v6-coder beats it by ~11pp. AIME 63.33 is far above any published 14–22B coder AIME.
• GPQA-Diamond 67.17 / IFEval 92.00: GPQA materially above Phi-4 (56.1) and Qwen2.5-14B (40.9). IFEval 92 beats Phi-4 (63.0) and Qwen2.5-14B (80.0).

v6-coder Q4_K_M vs Qwen3.6-35B-A3B-UD-IQ3_XXS — iso-disk 10-bench head-to-head

A whole-suite comparison at matched disk footprint: v6-coder Q4_K_M (13.24 GB / 5.09 bpw; 20.8B-total / 4B-active MoE) vs the Unsloth-Dynamic Qwen3.6-35B-A3B-UD-IQ3_XXS (14.07 GB / 3.44 bpw; 35B-total / 3B-active MoE). All 10 canonical templates, greedy T=0, through the omk_eval llama.cpp path; each model on its own validated build (v6-coder d6be315 with the 128e tokenizer; Qwen the af6528e #23643 MTP build). This is an iso-disk, cross-architecture read — a code-pruned 20.8B MoE vs a 35B/3B-active generalist at ~the same GB — not iso-bpw or iso-params. v6-coder Q4_K_M is the single plain, no-imatrix tier, i.e. its weakest-calibrated 4-bit, not a tuned imatrix build.

‡ Qwen's gsm8k / math500 are the Qwen-tuned-template scores. On the vanilla canonical templates Qwen posts 0% / 4% because Qwen3.x verbalizes the few-shot delimiter ("Question:" / "Problem:") inside its CoT and trips the lm-eval stop → empty content. Re-running with the redundant stop dropped (same few-shot, greedy, thinking budget; chat turn closes on <|im_end|>) recovers the true 98% / 98%. Gemma-4 CoT never emits the delimiter, so v6-coder's canonical numbers are unaffected. v6-coder GPQA here is 65.66 (Q4_K_M, plain) vs the 67.17 Q6_K headline elsewhere on this card — different quant tier, as expected.

Read:

• Code (the v6-coder remit) — v6 wins all four. MultiPL-E +18.00pp (88.33 vs 70.33), LCB-medium-55 +5.46, HE+ +3.05, HE +1.83. The smaller 20.8B code-pruned MoE beats the 35B generalist on every execution-scored code bench, at lower disk and on its weakest (no-imatrix) 4-bit tier.
• Math / science — Qwen wins. GPQA-D +11.61, AIME +13.33, gsm8k / math500 +6 / +9 (on the corrected templates). A full-breadth 35B generalist out-reasons the code-specialized prune on math-on-text and graduate science — expected, and the gap is the cost of the specialization.
• General — a wash. IFEval (92 vs 93) and ARC-Challenge (96.08 vs 96.84) are within a point.

Bottom line: at the same ~13–14 GB on disk, choose v6-coder Q4_K_M for code generation (it leads HE / HE+ / MultiPL-E / LCB) and Qwen3.6-35B-A3B for math / science reasoning. The two are complementary at this footprint, not strictly ranked — which is the point of the iso-disk framing.

---

Same-Stack GGUF HE+ Sweep — v6-coder vs Qwen2.5-Coder-14B-Instruct

Head-to-head HumanEval+ (164q, chat-aware shadow task) on identical hardware (single RTX 3090 24 GB) and identical eval recipe (llama-server -c 32768 -ngl 99 --parallel 2, omk_eval llama backend, lm-eval humanevalplus_full, greedy T=0). Qwen GGUFs are bartowski's Qwen2.5-Coder-14B-Instruct-GGUF. The "Comparison" tables above use paper-reported numbers; this section is what the same rig and same scorer actually measure. v6-coder per-tier HE+ is the Available Quantizations table above (complete).

Qwen2.5-Coder-14B-Instruct (14.7B dense) — bartowski quants

Qwen sits at 83–85% across the whole tier ladder. The paper-reported 87.2 HE+ is ~2pp above what bartowski's GGUFs deliver on this stack — a known llama-server chat-template vs vLLM-temp=0 quirk, not a quant defect.

Head-to-head by file size (v6-coder runs lower bpw at the same disk)

Pairing by tier name is misleading — v6-coder is a 20.8B-total MoE and Qwen is a 14.7B dense, so the same tier name maps to different file sizes. The fair comparison is iso-disk: at a given GB budget, which model wins HE+? At every band v6-coder uses 1.5–3.5 bpw less than Qwen and still scores higher (all tiers landed).

Reads:

1. A 92.68% cluster over a wide 92.07% plateau. The imatrix rebuild lifted the K-quants into a 92.68% top cluster — Q3_K_M (10.51 GB), Q4_K_S (12.21 GB), Q4_K_L (13.42 GB), Q5_K_M (15.07 GB) — sitting just above a broad 92.07% plateau (IQ3_XS 9.22 GB up through IQ4_XS / Q4_K_M / Q5_K_S / IQ3_M / Q3_K_L / Q3_K_XL). The 0.61pp steps are within HE+'s 1-question granularity, so pick by disk: Q3_K_M (10.51 GB) is the smallest tier in the top cluster, IQ3_XS (9.22 GB) the smallest on the plateau, IQ4_XS (11.01 GB) the safe 4-bit default.
2. vs Qwen, every band wins by +4 to +9pp at 1.5–3.5 bpw less. The biggest gap is the iso-disk ~10.5 GB point — v6 Q3_K_M (10.51 GB / 4.04 bpw / 92.68%) vs Qwen Q5_K_M (10.51 GB / 5.72 bpw / 83.54%): +9.14pp at the exact same file size, −1.68 bpw.
3. Sub-8 GB code-grade. IQ2_S (7.83 GB / 3.01 bpw / 89.02%) beats Qwen's IQ4_XS (8.12 GB / 4.42 bpw / 84.76%) by +4.26pp at smaller disk and lower bpw. IQ2_XS (7.77 GB) is the 2-bit cliff at 70.73% — use IQ2_S.
4. Sweep complete; imatrix lifted the low-bit K-quants most. Q8_0 / Q6_K top the cohort at 93.29%; the imatrix rebuild moved the 3-bit K-quants up sharply (Q3_K_M 85.98 → 92.68, Q3_K_S 82.93 → 87.80, Q3_K_L/XL 88.41 → 92.07), so a 3-bit tier (Q3_K_M, 10.51 GB) now leads the value frontier. The 4 CD-* contrib-dynamic tiers fill the 10–15.5 GB bands (CD-Q6_K 92.07, CD-Q4_K_M / CD-IQ4_K_M 91.46, CD-Q5_K_M 90.85). Below 9 GB the plain IQ ladder leads (IQ3_XS 9.22 GB / 92.07%, IQ2_S 7.83 GB / 89.02%).

CD recipes are open-source — generator at omnimergekit/scripts/generate_cd_maps.py.

---

Original Model Card

Gemma 4 A4B 98-Expert v6-coder (C6v3lcb) — LCB-targeted code prune (~20.8B)

Eval complete (Q6_K / llama.cpp). The full canonical 9-bench suite plus the extended LCB-medium-100 and MultiPL-E-100 code benches are filled below, every cell read from summary.json (greedy, cohort-pinned recipe). The 128e and 98e-v5-coder anchor columns are the matching Q6_K reference runs. NVFP4A16 is a deployment format and is not separately benchmarked (cohort policy).

Headline — the LCB targeting worked. LCB-medium-55 96.36% (+10.91pp vs v5-coder, and +1.81pp past the unpruned 128e), closing the −9.10pp hole that motivated the recipe; MultiPL-E macro 88.0% (+7 vs v5-coder, ≈128e); AIME recovers +10pp (53.33 → 63.33). The budget is paid on the non-code axes the LCB-only class weights deprioritized (MATH −3, IFEval −2 vs v5-coder).

A research checkpoint that prunes the unpruned Gemma 4 26B-A4B-it (128 experts, top-8 + shared, 30 layers) down to 98 experts per layer using a drop map that is the most code-faithful member of the v6-coder family: v5-coder's gentle C6 layer-relevance-weighted v4-floor recipe, re-derived on the corrected v3 code-pass calibration data, then steered specifically at LiveCodeBench-medium — the one code bench where expert pruning hurt most (−9.10pp vs 128e on v5-coder). Same 98e shape, same router, same attention, same norms as the rest of the cohort, plus the mandatory shared-FFN α=1.2 upweight all coder variants carry.

This is the head of the C6 → C6v3 → C6v3lcb line: each step holds the gentle C6 recipe fixed and changes exactly one variable (data, then target weighting), so the deltas are attributable.

Quantized formats

Eval is llama.cpp / Q6_K only for this cohort. NVFP4A16 is published as a vLLM-deployable format but is not benchmarked separately — the Q6_K scoreboard below is representative of the model's quality.

At a glance

---

Recipe

The drop map is produced by generate_drop_map_multiclass.py (omnimergekit) from per-expert, per-class contribution scores, then applied with expert_drop.py, then the shared expert is upweighted. Four design choices define C6v3lcb; each is held constant from v5-coder except where noted.

1. C6 gentle base recipe (unchanged from v5-coder)

The aggregation that ranks experts for dropping:

strategy      = max              # per-expert score = MAX over classes (not mean/geomean)
normalize     = rank             # rank-normalize within each (layer, class) before aggregating
protect_top   = 16               # the 16 highest-scoring experts/layer are never dropped
alpha         = 2.0              # contribution sharpening exponent
breadth_bonus = 0.5              # reward experts active across many classes (anti-overfit)
v4_floor_map  = v4_layer_floor_map.json   # per-layer floor: never drop below v4's keep on protected layers
baseline      = teacher_force_98e_p16_clean.json   # tie-break anchor

strategy=max + breadth_bonus is the load-bearing pair: it favors experts that are strongly useful to at least one class and broadly useful across classes, rather than experts with a high average that no single task depends on. This is the optimizer-off-manifold lesson encoded as a recipe — max/percentile beats mean/geomean for importance.

2. v3 calibration data + outlier fix (changed from v5-coder)

v5-coder ranked experts using the _fixed code-pass traces, which turned out to be ~86% NaN in the deep layers — accidentally NaN-blind where the model has real signal. v6-coder uses the corrected expert_neuron_v5_code_v3.json (real deep-layer signal restored, T73.0 fp32-hot-path patch), and scrubs the residual bf16 weight-norm artifacts with a median-based outlier clamp:

data                = expert_neuron_v5_code_v3.json
outlier_mode        = median
outlier_wnorm_thresh = 1e4       # clamp expert weight-norms above 1e4 to the layer median

Calibration corpus (v5_code_pass_traces.json, 360 traces): Tier-A 128-token comprehension prompts (342 traces, 1×) + Tier-B 2048-token windowed pass-traces (18 traces, 3×) — the long-window tier captures sustained code reasoning, which is where pruned variants ruminate.

3. LCB-only targeting — the "lcb" in C6v3lcb (changed)

Eight contribution classes are scored; the class weights steer which specialists are protected. v6-coder zeroes the HumanEval targeting and concentrates the targeted budget on LiveCodeBench-medium:

Rationale: HE/HE+ were already at/above 128e on v5-coder (+1.22 / +1.22), so spending protection budget on them is wasted; LCB-medium was the −9.10pp hole. Removing the HE targeting frees the floor to protect LCB-relevant experts harder. The resulting map is 95% identical to C6v3 (generic_code 3× dominates), so HE+/IFEval are expected at ≈C6v3 — the bench that tests the hypothesis is LCB-medium.

4. Mandatory shared-FFN α=1.2 (cohort rule)

After expert drop, router_shared_upweight.py --alpha 1.2 --target mlp.down_proj.weight scales the always-on shared expert's down-projection by 1.2×. Without it the pruned model is the "weak/ruminating" pre-T18 baseline; every coder variant carries it or the v5-coder comparison is unfair. Verified by the .shared_applied marker in this repo.

---

Expert mapping

• Uniform 30 experts dropped per layer, all 30 layers → 98 kept/layer (128 → 98).
• Mean overlap with the teacher-force baseline: 17.8 / 30 dropped per layer (min 11, max 24). The recipe re-ranks ~12 of 30 dropped slots/layer away from the naive teacher-force drop — that re-ranking is where the code-faithfulness lives.
• Aggregated keep scores (rank-normalized, α=2.0, breadth-bonus) span ~0.0–3.5 per layer with mean ~1.85; the protected top-16 sit at the high end, the dropped 30 at the low end, with boundary_ties_within_1pct 2–39/layer (highest in the shallow layers 0/10 — those layers have the flattest expert-importance profile).

Full per-layer keep lists are in expert_drop_metadata.json (per_layer_keep) in this repo; the ranking provenance (per-layer agg min/max/mean, ties, overlap-vs-baseline, v5-only non-overlap experts) is in scripts/v6coder_C6v3lcb_drop_map.json.summary.json.

---

Problems solved

1. Rumination on hard code/reasoning problems. Pruned Gemma 4 variants fall into <think> loops on the hardest LCB / GPQA / AIME problems, blowing the token budget without converging. C6v3lcb attacks this two ways: (a) the LCB-targeted drop map keeps the experts that the hard-problem pass-traces actually use; (b) at eval time the lcb_medium_*_v4 template applies a thinking_token_budget=12288 forcing function (parser=gemma4 + enable_thinking) that caps the rumination loop. T109 rumination-signal gate (GPQA-48 / AIME probes) confirmed the variant generates and converges rather than looping.
2. Data-vs-recipe disentanglement (T102). v6-coder isolates whether v3's earlier regressions were the data or the recipe: holding v5-coder's gentle C6 recipe fixed and only swapping in v3 data + the outlier fix. C6v3 ≈ v5-coder in smoke → the v3 data is fine and C12's aggressive recipe was the rumination cause; C6v3lcb then re-steers that clean baseline at LCB.
3. Deep-layer calibration corruption (T73.0). The _fixed traces were ~86% NaN in deep layers; v6-coder uses real-signal v3 data with a median outlier clamp so deep-layer experts are ranked on genuine contribution, not NaN-blindness.
4. Unfair-comparison trap. The shared α=1.2 step is mandatory and marker-verified, so v6-coder vs v5-coder is apples-to-apples.

---

Scoreboard — Q6_K GGUF, llama.cpp, greedy

Full 9-bench llama.cpp Q6_K run, llama-server --reasoning-format deepseek --reasoning-budget 12288 --parallel 2, greedy (T=0, top_p=1, top_k=0, do_sample=false). The 128e and v5-coder columns are the bartowski-Q6_K / v5-coder-Q6_K reference runs under the identical recipe (apples-to-apples within the llama.cpp/Q6_K backend).

Read: v6-coder lands on/above v5-coder on 8 of 9 benches and beats the unpruned 128e on every code bench (HE +2.44, HE+ +2.44, LCB-55 +1.81). The headline is LCB-medium-55 +10.91pp vs v5-coder — the targeted hole is not just closed but pushed past the base model. AIME recovers +10pp (53.33 → 63.33). The cost lands on MATH (−3) and IFEval (−2) vs v5-coder — the non-code generalist axes the LCB-only class weights (1 1 3 1 1 0 0 2) deliberately deprioritized.

Extended code benches — LCB-medium-100 + MultiPL-E-100 (Q6_K, llama.cpp, greedy)

Run on solidpc (T112), same Q6_K / greedy recipe; MultiPL-E scored via the nuprl/multipl-e-evaluation Docker image. 128e and v5-coder columns from the v5-coder card.

LCB-medium-100 (100-problem superset of LCB-medium-55 v4):

(v6-coder also clears 128e on LCB-100 by +1.00pp — the LCB-targeting win holds on the 100-problem superset, not just the 55-problem v4 set.)

MultiPL-E-100 (HumanEval → Rust / Java / JS, 100/lang, chat-mode + code extraction):

v6-coder near-fully recovers MultiPL-E to the 128e level (macro −1.67pp) from v5-coder's −8.67pp gap — code generalization in non-Python languages tracks the LCB-targeting win, not just the in-distribution LCB benches. (264/300 passed; micro = macro = 88.0%.)

Coder-field comparison — v6-coder vs Qwen2.5-Coder-14B / 7B + Qwen3.5-9B (Q6_K, llama.cpp, greedy)

The 9 canonical benches + MultiPL-E-100, all on the identical llama.cpp Q6_K / greedy recipe (reasoning models served with --reasoning-format deepseek --reasoning-budget 12288 --parallel 2). Architectures differ — this is a same-harness comparison, not a same-class one:

• v6-coder — Gemma-4 26B-A4B MoE pruned to 98 experts (~20.8B total, ~A4B active), reasoning.
• Qwen2.5-Coder-14B / 7B-Instruct — dense, non-reasoning code specialists (bartowski Q6_K).
• Qwen3.5-9B — dense reasoning model (bartowski Q6_K).

† Qwen2.5-Coder-14B HumanEval+ / LCB-medium-55 reused from the same-stack GGUF HE+ sweep (not re-run in this chain). All comparison cells filled 2026-05-28 (9B Fix-A re-eval + 14B Q6_K reference). running legend: eval in flight — table updated as results land.

Note on Qwen3.5-9B. Qwen3.5-9B is a verbose, slow thinking model: it emits long <think> reasoning chains (often ≥1900 tokens even on a trivial GSM8K question), so it runs several× slower per question than the non-reasoning Qwen2.5-Coder models — well beyond what its 9B size would suggest. Its GSM8K / MATH-500 / GPQA cells are being re-run after a harness fix: under batched, reasoning-parsed serving the verbose thinking intermittently left the final answer inside the reasoning block (mis-scored as empty content). ARC (96.76%) and AIME (56.67%) were unaffected.

---

Answer-length analysis (anti-rumination)

v6-coder thinks longer on average than 128e on the code benches — it spends the full bounded thinking_token_budget=12288 where 128e often answers in a few hundred tokens. The question this section answers is whether that extra length is rumination (long thinking that fails to reach an answer — the failure mode the LCB targeting was built to fix) or productive reasoning that lands on a PASS. The tables below compare per-problem completion length against 128e and v5-coder on the same problems, from omk_eval token_stats (characters from the raw completion text; tokens via the 128e tokenizer), on the real-n benches. 128e and v5-coder are the matching Q6_K runs. The short answer: by the rumination-as-wasted-thinking definition, v6-coder ruminates less than 128e, not more — 128e's longest LCB outputs (up to 51k chars) are failures, while v6-coder's long outputs overwhelmingly pass.

Per-problem completion length — characters (p50 / p90 / max):

Per-problem completion length — tokens (p50 / p90 / max):

Budget-saturation incidence — share of problems whose completion reached ≥12k tokens (at/near the thinking_token_budget=12288 cap). Saturation by itself is not rumination: a saturated output that reaches a correct PASS is productive use of the budget. The pruned variants saturate on nearly everything; 128e almost never does — but see the next table.

Rumination — long thinking that fails to PASS. The right metric is not median length (128e looks short only because it answers easy problems fast). It is the share of the model's budget-saturated outputs that still fail — i.e. tokens burned without a correct answer:

Every 128e output that reaches the budget cap is a failure (100%), and 128e's failed problems run 5.7× longer than its passed ones (mean 8368 vs 1468 tok): 128e only thinks long when it is lost. v6-coder thinks long on ~97% of problems but 96% of those long outputs PASS — its long thinking is overwhelmingly productive, not rumination.

Key findings:

• LCB — v6-coder ruminates less than 128e, under the only definition that matters (long thinking that fails to reach a PASS). The naive read of the median says the opposite — 128e's median LCB completion is ~0.6–1.2k tokens vs v6-coder's budget-capped ~12.7k — but that short median is an artifact of mixing easy and hard problems. 128e only thinks long when it is lost, and then it fails: every 128e output that saturates the budget is a failure (2/2 on LCB-100, 1/1 on LCB-55), and the single longest output across all three models — 51,568 chars / 15,585 tokens on lcb/leetcode/3659 — is a 128e failure, one that both v6-coder (PASS, 12,691 tok / 32,676 chars) and v5-coder solve in two-thirds the length. v6-coder, by contrast, saturates the budget on ~97% of problems but 96% of those long outputs PASS; its longest LCB-55 output is a correct answer and its max length (41–43k chars) sits below 128e's 51k. Wasted long-thinking rate: 128e 100%, v6-coder ~4%. And v6-coder beats 128e on score on both axes (96.36 / 96.00 vs 94.55 / 95.00) — more correct and a lower worst-case tail.
• 128e's routing is erratic in both directions, and the targeted map repairs both. 128e fails LCB two ways: by over-thinking (3659: 51k chars, no answer) and by under-thinking (lcb/leetcode/3000: a premature wrong answer in 579 tokens; 3793: 3101 tok) — on the very same problems v6-coder spends the full ~13k-token budget and passes. This bidirectional instability is the fingerprint of an imperfect base router; the 51k outliers and the lower 128e score are the symptom. v6-coder's LCB-targeted expert map corrects 3 of the 5 problems 128e fails on LCB-100 (2 of 3 on LCB-55). The thinking_token_budget=12288 forcing-function only bounds the loop; the targeted map is what makes the bounded thinking land on a PASS instead of a 51k-char dead end. (Versus v5-coder, v6-coder is statistically the same length — median 12799 vs 12804 tok on LCB-55 — yet passes far more: 53/55 vs 47/55, 96/100 vs 91/100.)
• AIME — genuinely less rumination, +10pp. Median completion 939 tok vs v5-coder's 1156, p90 char tail 8404 < 9239, scoring 63.33% vs 53.33% — shorter chains and more correct.
• GSM8K — tightest of the three (max 396 tok; 128e 1107, v5-coder 858); no rumination tail.
• IFEval — no ruminating outlier (max 7200 chars vs v5-coder's 17539; ≈128e 7425).
• GPQA — a base-model ceiling, not a prune artifact. p90 completion tokens peg at ~8189 on all three models; the hard-question rumination is inherited from Gemma 4 and is identical across the cohort, not widened by pruning.
• ARC — one shared single-sample outlier (~16266 tok / ~44k chars) on both pruned variants, absent on 128e; p50/p90 normal and identical, so not systemic length growth.
• MultiPL-E — most compact code of the three. On the bench where the answer is the code (extracted final block, no reasoning trace), v6-coder has the tightest tail — max 2169 chars vs 128e 2944 / v5-coder 3083, p90 591 chars / 184 tok — at ≈128e pass-rate (macro 88.0%).
• Net: v6-coder is more accurate than 128e on every LCB axis (96.36 / 96.00 / MultiPL-E macro 88.0) at a higher average inference cost (it thinks the full bounded budget) but a lower worst case (max 41–43k chars vs 128e's 51k) and a far lower wasted-thinking rate (~4% vs 100%). On the short-reasoning benches (AIME / GSM8K / IFEval) it additionally reduces length vs v5-coder. The targeted map never traded length for accuracy — it traded 128e's erratic, sometimes-catastrophic tail for a higher but bounded and productive average.

Methodology / data integrity. Per-problem lengths come from omk_eval token_stats over each bench's per-problem samples archive — lm-eval samples_*.jsonl for the 9-bench, and the native runners' lcb_result.samples.jsonl / mpe_result.samples.jsonl (mirrored into the durable sqlite resume cache) for LCB/MPE. MultiPL-E is tabulated, but its rows measure extracted-code length, not reasoning: its samples store only the final code block (no <think> trace, finish_reasons all terminal, thinking_tokens_est=0), so MPE is a code-conciseness reference — not a rumination metric — on which v6-coder is the most compact. The reasoning-bearing benches (LCB / GPQA / AIME / MATH) carry the rumination signal; MPE tokens there are tokenizer-estimated (the code-extraction path returns no server token count). thinking_tokens_est is 0 across all benches under llama.cpp --reasoning-format deepseek (reasoning is emitted inline, not as a separable block), so the completion-token columns are the length signal. The LCB/MPE token_stats are computed over per-problem rows keyed on doc_id (the native runners now emit it, fixed 2026-05-25) — earlier card revisions briefly collapsed these to n=1; all numbers here are at full n (LCB-55 n=55, LCB-100 n=100, MultiPL-E n=300).

---

Reproduce

# 1. drop map (omnimergekit)
python scripts/generate_drop_map_multiclass.py \
    --data scripts/expert_neuron_v5_code_v3.json --target 98 \
    --protect-top 16 --alpha 2.0 --strategy max --normalize rank \
    --breadth-bonus 0.5 --v4-floor-map scripts/v4_layer_floor_map.json \
    --baseline-drop-map scripts/teacher_force_98e_p16_clean.json \
    --outlier-mode median --outlier-wnorm-thresh 1e4 \
    --class-weights 1 1 3 1 1 0 0 2 \
    --output scripts/v6coder_C6v3lcb_drop_map.json

# 2. prune + mandatory shared upweight (see scripts/build_v6coder_C6v3lcb.sh)
python scripts/expert_drop.py --source-dir google/gemma-4-26B-A4B-it \
    --drop-map scripts/v6coder_C6v3lcb_drop_map.json --suffix=-v6-coder-C6v3lcb-it
python scripts/router_shared_upweight.py \
    --model-dir google/gemma-4-A4B-98e-v6-coder-C6v3lcb-it \
    --alpha 1.2 --target mlp.down_proj.weight

Lineage & provenance

• Base: google/gemma-4-26B-A4B-it (128e, fresh prune — not derived from v4/v5 weights).
• Cohort: v5-coder (C6 _fixed) → C6v3 (C6 v3-data) → C6v3lcb (this, LCB-targeted).
• Tooling: omnimergekit — expert_drop.py, generate_drop_map_multiclass.py, router_shared_upweight.py; eval via omk_eval.py + EVAL_PROTOCOL.
• Eval sampler is frozen GREEDY across the whole cohort for apples-to-apples comparison.

License: Gemma Terms of Use. This is a derivative of Gemma 4 and inherits those terms.