REBUTTAL-math-OPD-Expert8B-Qwen3-4B-Base-resp8192

REBUTTAL-math-OPD-Expert8B-Qwen3-4B-Base-resp8192

On-policy distillation (OPD) of Qwen/Qwen3-4B-Base from a frozen expert teacher Qwen/Qwen3-8B (thinking mode ON), trained with verl.

The teacher gets no privileged context (teacher_plain_prompt=True): it sees only the original question, and every sample participates. The student is pulled toward the teacher distribution via top-K logit Jensen-Shannon divergence on its own on-policy rollouts — no environment reward is used. This is the "just distill from the bigger model" control for the reward-anchored RLRT / RLSD arms.

Contents

Merged HF format — load directly with transformers or vLLM.

Training configuration

max_response_length=8192 is what distinguishes this run from the resp20480 variant — it is the knob this arm exists to move, and it is the generation cap that actually bounds rollout length (not max_model_len).

Evaluation

16 rollouts per prompt, thinking ON, temperature=0.6, top_p=0.95, top_k=20, max_tokens=30720 (the model's context is 32768 and it carries no rope scaling, so generation is capped there regardless of what max_tokens says).

Avg@16

Pass@16

Step 100 numbers are still running.

Read these next to the truncation rate. Because the policy was trained under an 8192-token cap and is evaluated at 30720, it frequently fails to terminate: 20.1% of MATH500 samples at step 50 hit the cap (other arms trained at 20480 hit it 1.4–3.3% of the time), and a capped sample emits no \boxed{} and scores zero. The accuracies above are therefore a lower bound.