Qwen3.8-27b-Terse-Coder-GGUF

Qwen3.8-27b-Terse-Coder-GGUF

GGUF builds of Shockem/Qwen3.8-27b-Terse-Coder (round 8, final) — a fine-tune of Qwen/Qwen3.8-27B with ~1/10 the chain-of-thought reasoning tokens on coding tasks and correctness preserved.

Quant spread (Q4_K_M smoke-tested on llama.cpp — correct code generation, reasoning parsing, ~22 tok/s on 2× RTX 5060 Ti):

Usage (llama.cpp server): Chat template: use Shockem/froggeric-terse-coder (based on froggeric/Qwen-Fixed-Chat-Templates) (also bundled here as chat_template.jinja) — it carries the anti-rumination tool rules the model was evaluated with; serving without it changes agentic behavior.

llama-server -m Qwen3.8-27b-Terse-Coder.Q4_K_M.gguf \
  --mmproj Qwen3.8-27b-Terse-Coder.mmproj-f16.gguf \
  -ngl 99 -c 32768

The MTP draft layer is included in the GGUF but unused by llama.cpp (harmless warnings about unused blk.N.nextn.* tensors are expected); speculative decoding is a vLLM-side feature. Behavior/quality reference: HE+ 91.5 / MBPP+ 79.4 / GSM8K 98.5 / GPQA 79.8 on the source repo (vLLM measurements; GGUF Q4_K_M verified by smoke test only — validate quality on your own hardware before relying on it).

Harness recommendations

How you drive the model matters as much as which build you run. Measured on Terminal-Bench 4.0 (Sep 2026, local vLLM serving): prompt-level guardrails did not reduce token burn or stop grinding — the dominant cost was the harness re-sending the full transcript every step. Zero tool-call loops observed in any trajectory.

• Let the server own sampling — omit temperature/top_p/top_k; the checkpoint config applies temp 0.6 / top_k 20 / top_p 0.95 / repetition_penalty 1.05. Never send min_p (vLLM rejects it under MTP spec decode).
• Serve with the bundled chat template (chat_template.jinja; also at Shockem/froggeric-terse-coder). The anti-rumination rules inject even alongside custom system prompts — bypassing the template (bare completions API, client-side template) is the "repeats already-done steps" failure mode.
• Per-turn max_tokens 4–8k is generous (routine coding answers are 25–150 tokens; thinks median ~38). Use 49152 only as a session floor for agents, not a per-turn target.
• Context management is the biggest lever. Keep ~3 recent tool results, cap older blobs at ~4k chars. Unbounded agent history was worth ~270k+ input tokens per task in our probe — that is a harness property, not a model property.
• Think text arrives in reasoning_content — empty content with non-empty reasoning_content is a normal turn, not an error.
• Raise reasoning_effort for genuinely hard problems — the model still scales deliberation up when the problem needs it (GPQA median ~900 reasoning tokens); terseness targets waste, not deliberation.

Source and provenance: fp16 merge at Shockem/Qwen3.8-27b-Terse-Coder; vLLM deployment build at Shockem/Qwen3.8-27b-Terse-Coder-NVFP4; adapter lineage at Shockem/Qwen3.8-27b-Terse-Coder-LoRA. Licensed Apache 2.0, same as the base model (© Qwen Team, Alibaba Cloud).