Qwopus3.6-27B-v4

Qwopus3.6-27B-v4

A reasoning-enhanced version of Qwen3.6-27B, fine-tuned with structured Chain-of-Thought (CoT) distillation data to significantly improve mathematical reasoning and competition-level problem solving.

Model Description

Qwopus3.6-27B-v4 is obtained by supervised fine-tuning Qwen3.6-27B on a curated CoT reasoning corpus derived from Claude Opus extended thinking traces. The training follows a curriculum learning strategy — progressively scaling from short to long reasoning chains — to develop deep, structured reasoning without collapsing the model's instruction-following abilities.

The model produces explicit <think>...</think> reasoning traces before giving final answers, enabling transparent, step-by-step problem solving on competition mathematics and graduate-level science.

Benchmark Results

AIME 2025: 86.67% — a +30 point improvement over the base model, demonstrating strong gains in competition-level mathematical reasoning through CoT distillation.

Note: Degradation on HMMT/GPQA/IFBench is attributed to distribution shift from the math-focused training corpus. General instruction-following and multi-domain reasoning are expected to improve in future versions with mixed-domain replay training.

Training Details

Training Data

Trained on CoT-Trace-Inverted-28K — an openly released CoT distillation dataset constructed and curated by the author.

The dataset was built through a multi-stage pipeline:

1. CoT Collection: Extended thinking traces captured from Claude Opus 4.5/4.6/4.7 via API
2. Trace Inversion: Compressed reasoning bubbles reconstructed into full reasoning chains using Trace-Inverter-4B
3. Curriculum Sorting: Samples globally sorted by reasoning chain length for progressive training
4. Quality Filtering: NLP surface tasks, placeholder samples, and short-think samples removed

The dataset contains 28K high-quality CoT samples spanning mathematics, science, coding, and logical reasoning, and is publicly available at: 👉 https://huggingface.co/datasets/skiboyvv/CoT-Trace-Inverted-28K

Curriculum Learning Strategy

Training follows the three-stage curriculum from the original Qwopus research:

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "skiboyvv/Qwopus3.6-27B-v4"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="bfloat16", device_map="auto")

messages = [
    {"role": "user", "content": "Find all integer bases b > 9 for which 17_b divides 97_b, and compute their sum."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

vLLM Deployment (Recommended)

vllm serve skiboyvv/Qwopus3.6-27B-v4 \
  --served-model-name qwopus-v4 \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --dtype bfloat16 \
  --enable-prefix-caching \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

Limitations

• Mathematical reasoning (AIME-style) is significantly enhanced, but general instruction-following degrades compared to the base model.
• Not recommended as a drop-in replacement for general-purpose tasks.
• Best suited for competition math, scientific reasoning, and structured problem solving.

Citation

@model{qwopus_v4_2025,
  author    = {Liu Jinhao},
  title     = {Qwopus3.6-27B-v4: CoT Distillation Fine-tuned Qwen3.6-27B},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/skiboyvv/Qwopus3.6-27B-v4}
}

---

Model trained and released by Liu Jiahao. CoT distillation pipeline, curriculum design, and training configuration are original contributions of the author.