ReflectRL-Qwen2.5-3B-Instruct-GRPO

ReflectRL-Qwen2.5-3B-Instruct-GRPO-ReflectRL

This repository contains the checkpoint for ReflectRL-Qwen2.5-3B-Instruct-GRPO-ReflectRL, presented in the paper ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning.

• GitHub Repository: ReflectRL
• Base Model: Qwen/Qwen2.5-3B-Instruct

Overview

ReflectRL is a lightweight framework for learning from Golden Negative Trajectories (GNTs) during on-policy post-training. Instead of imitating failed expert trajectories directly, ReflectRL uses them as reflective context during training and gradually transitions the policy back to direct reasoning for inference.

Motivated by the Reflection Advantage, ReflectRL allocates part of each rollout group to a reflective interface during training and gradually decays that allocation to zero, allowing the final model to be used directly with standard prompts.

Citation

@article{bi2026reflectrl,
  title={ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning},
  author={Bi, Jinhe and Zhou, Chennan and Jin, Zengjie and Aniri and Lu, Shuo and Huang, Wenke and Cao, Hu and Xiao, Xun and Zhu, Zhihong and Tresp, Volker and Shen, Fei and Ma, Yunpu and Chua, Tat-Seng},
  journal={arXiv preprint arXiv:2608.03972},
  year={2026}
}