pepo-task5-qwen2.5-vl-7b-step504

PEPO (paper-aligned Task-5 reconstruction) — Qwen2.5-VL-7B, ViRL39K, step 504 / 606 (last checkpoint before the late-epoch-2 entropy collapse; the run degenerated after ~step 530)

Research checkpoint: a reconstruction of the PEPO baseline at 7B (the original paper reports 3B/2B only). Provided as-is for comparison purposes.

• Method: PEPO (Perception-Exploration Policy Optimization, arXiv 2603.22847), official ms-swift implementation (xzxxntxdy/PEPO), method constants as released: gate_alpha 0.1, gate_temperature 1.8, cosine perception prior, vision weights on.
• Training config = paper Appendix A Table 1 Task 5 (ViRL39K): 128 prompts/step x 8 rollouts (1024 completions), 2 epochs (606 steps), lr 1e-6 constant, KL beta 0.01, completion cap 1024, min/max pixels 200704/1003520, freeze ViT, reward 0.1format + 0.9accuracy (\boxed{}, mathruler grader, PAPO-equivalent).
• 7B memory adaptations (do not change the optimization math): micro-batch 1 x grad-accum 128, DeepSpeed ZeRO-3, vLLM colocate 0.35 + sleep_level 2 + offload. NOTE: PEPO reports 3B/2B only; this is our 7B extrapolation.
• Answer protocol: <think> ... </think> then \boxed{}. Eval protocol (family table): avg@8, temp 1.0, top_p 1.0, 2048 new tokens.
• Weights: bf16 safetensors exported from ms-swift checkpoint-504 with config flattened to the pre-4.52 transformers layout (weight tensor names unchanged) so vLLM/transformers 4.51 can load it.