Qwen3.8-27B-Opus-Distill-v2

Qwen3.8-27B-Opus-Distill-v2

Qwen/Qwen3.8-27B fine-tuned (LoRA, merged) on verified-genuine Claude Opus reasoning traces. This release fixes a reasoning-loop bug reported against the first release (barozp/Qwen3.8-27B-Opus-Distill, "v1") while keeping the reasoning-quality gains v1 was known for.

Why this release exists

A community member (zxbc2023) reported a clean, deterministic bug on v1: certain stacked output-format constraints (most notably "no prose" combined with a second constraint like "no markdown") sent the model into a non-converging self-verification loop — burning the entire token budget on repeated "wait, let me re-check…" passes with zero visible output. Their repro was exact and reproducible at temp=0, and they correctly diagnosed it as likely inherited from the distillation data rather than a capability gap. Full credit to them for the report that made this fix possible.

Root cause. Investigating v1's training data (opus-reasoning-distill-train) showed that 83% of it had actually been sourced from Jackrong's TraceInversion datasets — reasoning traces reconstructed by a small auxiliary model (Trace-Inverter-4B) from a compressed summary, not the teacher model's real output (openly documented on those datasets' own cards). That reconstruction process appears to have baked in a rigid "enumerate constraints, then re-verify" template — plausibly both the source of v1's reasoning-quality gains and the loop bug, two sides of the same mechanism.

The fix, in two steps:

1. First attempt: retrain purely on a genuine-trace dataset (lordx64/reasoning-distill-claude-opus-4-7-max). This fixed the loop bug completely but collapsed the reasoning-transfer gain (GPQA-Diamond dropped back to base/random level) — the domain mix mattered as much as trace authenticity.
2. This release: reconstructed v1's exact prompt distribution, but replaced every row traced back to Jackrong's fabricated sets with the verified genuine trace for the same prompt (from lordx64 or Roman1111111/claude-opus-4.6-10000x). Then ran a full-dataset scan (30 parallel LLM-judge passes over all 14,250 rows) for the specific pathological pattern and removed every flagged row (169) plus every row whose source was never verified against a known-genuine dataset (2,429). Net: 14,250 → 11,716 rows, 100% traced to verified sources.

Bug fix validation

Re-ran zxbc2023's exact repro (temp=0, seed=1234, fib(n) with stacked constraints) against this model:

All three converge quickly with </think> closing normally — no loop, no budget exhaustion.

Benchmark results

Measured with lm-evaluation-harness, 0-shot, loglikelihood (multiple-choice), chat template OFF, QUICK mode (--limit 500) — identical protocol to v1, so the Δ column is the meaningful signal.

Reading the table: knowledge (MMLU) and language modeling (wikitext) are unaffected; reasoning (ARC, GPQA) shows a large, real gain — slightly smaller than v1's (expected: v1's gain was partly driven by the same fabricated content that caused the bug), but still far above noise. GPQA in particular is a same-protocol delta vs. base, not comparable to Qwen's own published 89.2 (thinking-mode-on, different harness) — see v1's card for the full caveat.

Model overview

• Base model: Qwen/Qwen3.8-27B — dense 27B, native vision-language, native MTP.
• Method: LoRA (r=64, alpha=64, dropout=0.05), merged into the base weights.
• LoRA targets: attention q/k/v/o_proj on the 16 full-attention layers only; FFN gate/up/down_proj on all 64 layers.
• Training data: barozp/opus-reasoning-distill-v2 (11,716 examples, 100% verified-genuine Opus traces).
• Training run: 1 epoch (696 steps), lr=1e-4 cosine with 3% warmup, effective batch 16, MAX_SEQ=4096, bf16, ~4h15m on an A100 80GB.
• Final validation loss: 0.4334 — lower than both v1 (0.4647) and the intermediate pure-genuine attempt (0.805), despite less data.
• Vision + MTP: carried over from base, never trained.

Quick start

from transformers import AutoModelForImageTextToText, AutoProcessor
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "barozp/Qwen3.8-27B-Opus-Distill-v2", dtype=torch.bfloat16, device_map="auto",
)
processor = AutoProcessor.from_pretrained("barozp/Qwen3.8-27B-Opus-Distill-v2")

Text-only inference also works via AutoModelForCausalLM.

Related

• Qwen3.8-27B — the base model.
• barozp/Qwen3.8-27B-Opus-Distill — v1. Still works well for most use cases; this release specifically fixes the stacked-constraint reasoning-loop edge case.
• barozp/opus-reasoning-distill-v2 — the training dataset, with full provenance notes.
• Thanks to zxbc2023 for the bug report that led to this release.

Quantized releases of this checkpoint:

• barozp/Qwen3.8-27B-Opus-Distill-v2-FP8 -- block-wise FP8 (e4m3, 128x128) for CUDA serving stacks; paired BF16-vs-FP8 evals show deltas within noise.
• barozp/Qwen3.8-27B-Opus-Distill-v2-GGUF -- llama.cpp / Ollama / LM Studio quants (BF16 down to IQ1_M), mmproj vision, MTP self-speculative decoding.
• barozp/Qwen3.8-27B-Opus-Distill-v2-MLX-4bit and MLX-8bit -- Apple Silicon mlx-vlm conversions.