PCSS-Qwen3.5-9B

PCSS-Qwen3.5-9B

GitHub Repository | Blog Post | Reproduction Notebook

PCSS-Qwen3.5-9B is an experimental checkpoint demonstrating that fine-tuning small base models on pure logic deduction traces can elicit substantial out-of-domain reasoning generalization. Starting from Qwen/Qwen3.5-9B-Base, this model was fine-tuned on just 500 5x5 zebra puzzles from the tamewild/instruct5 dataset, which contains zero mathematical data.

For fine-tuning on these long-form logic traces, the model was trained using PCSS (Per-Example Calibrated Sigmoid Scaler), an adaptive loss scaler derived from KTO. The training process leveraged MiSS (Matrix Shard Sharing) for parameter-efficient fine-tuning (PEFT) alongside a preliminary PCSS extension for Multi-Token Prediction (MTP) with k=1, completing in approximately 40 minutes on a single NVIDIA H200 NVL GPU. (Note: The accompanying reproduction notebook was run on an H100 SXM, producing comparable results but with minor variance due to training non-determinism).

---

Chat Template & Evaluation Mode

• Control Token Pre-training: As noted on the Qwen/Qwen3.5-9B-Base model page, the base checkpoint was released with chat template control tokens (such as <|im_start|> and <|im_end|>) explicitly pre-trained "to allow efficient LoRA-style PEFT with the official chat template," mitigating the need to fine-tune embeddings.
• Non-Thinking Mode: The official Qwen 3.5 chat template includes a built-in toggle (enable_thinking) that controls whether the model generates a chain of thought within <think> tags prior to its final response. In our experiments, we only train and evaluate in non-thinking mode (enable_thinking=False). This ensures the untrained base, our fine-tuned checkpoint, and the official post-trained model are evaluated under strictly comparable conditions without generation within these tags.

---

Benchmark Results

Models were served via vLLM (0.19.1) using the following configuration:

• Max Completion Tokens: 16,384
• Sampling Parameters: temperature=1.0, top_p=0.95, top_k=20
• Presence Penalty: 0.0 for the untrained base and our fine-tuned model (1.5 was applied for the official post-trained checkpoint as recommended by Qwen).

To account for sampling variance, most benchmarks were evaluated using multiple independent samples. All reported scores are Pass@1 estimates. Pass@3 estimates are only reported for benchmarks with sufficient sample sizes (n >= 6).

---

Training Configuration & Hyperparameters

For full training details and mathematical derivations, please refer to the Reproduction Notebook and Blog Post.

• Base Model: Qwen/Qwen3.5-9B-Base
• Dataset: tamewild/instruct5 (500 5x5 zebra puzzles)
• PEFT Method: MiSS (Matrix Shard Sharing), rank r=512
• Loss Function: PCSS (beta=0.65, peak_scale=5.0)
• Optimizer: 8-bit AdamW (LR: 5e-6, beta2: 0.99994, Weight Decay: 0.01)
• LR Scheduler: Constant learning rate with 1-epoch linear warmup
• Training: 5 epochs at batch size 1

---

Usage & Inference

Serving with vLLM

The checkpoint bundles the pre-configured chat_template.jinja. To serve the model in non-thinking mode with native MTP speculative decoding:

# Note: Configured to 2 tokens for better throughput during our benchmarking, despite k=1 fine-tuning.
vllm serve tamewild/PCSS-Qwen3.5-9B \
  --max-model-len 32000 \
  --language-model-only \
  --generation-config vllm \
  --host 127.0.0.1 \
  --port 18000 \
  --gpu-memory-utilization 0.90 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

Prompt Formats

All evaluations were performed without a system prompt. The following zero-shot Chain-of-Thought templates were used (note the specific divergence for zebra puzzles, where the official post-trained model was evaluated using a prompt that requested a markdown table):

1. Mathematical Benchmarks (MATH-500, Full MATH, AIME 2025, AIME 2026, ArXivMath)

{problem}

Please reason step by step, and put your final answer within \boxed{}

2. Multiple-Choice Benchmarks (GPQA Diamond, MMLU Redux)

{problem}

Please reason step by step, and put your final letter choice within \boxed{\text{}}

3. Logic Grid Benchmarks (Zebra Puzzles)

For the Untrained Base and PCSS Fine-Tuned Models:

{problem}

Provide the solution grid as the final answer.

Please reason step by step, and put your final answer within \boxed{}

For the Official Post-Trained Model:

{problem}

Provide the solution table as the final answer.

Because the official post-trained model struggled to consistently output parsable LaTeX grids, its prompt was adjusted to request a markdown table. While this adjusted prompt omitted the explicit Chain-of-Thought instruction, the model natively outputs extensive step-by-step reasoning traces on these puzzles regardless. The table below contextualizes the generation lengths (in tokens) alongside benchmark accuracy across the 5x5 and 6x6 zebra puzzles.