Downloads · 30 days
25
3% of all-time downloads
wenlongzhao/ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_paper-config
ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_paper-config is a text generation model from wenlongzhao. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Full-parameter SFT of Qwen/Qwen2.5-3B-Instruct, following the cold-start recipe in Appendix E.2 of Agentic Reinforced Policy Optimization (ARPO).
Downloads · 30 days
25
3% of all-time downloads
All-time downloads
922
Public
Parameters
3.1B
6.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
Full-parameter SFT of Qwen/Qwen2.5-3B-Instruct, following the cold-start recipe in Appendix E.2 of Agentic Reinforced Policy Optimization (ARPO).
This is the paper variant (max length 4096, global batch 128, weight decay 0.1). A companion code variant matching the released ARPO LLaMA-Factory yaml is at wenlongzhao/ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_code-config.
Trained on dongguanting/ARPO-SFT-54K (Tool-Star 54K + STILL), the official ARPO SFT mix.
Trained with LLaMA-Factory full SFT, DeepSpeed ZeRO-3, BF16, and FlashAttention-2.
| Hyperparameter | Value |
|---|---|
| Learning rate | 7e-6 |
| LR schedule | cosine, warmup ratio 0.1 |
| Epochs | 3 |
| Global batch size | 128 |
| Max sequence length | 4096 |
| Weight decay | 0.1 |
| Precision | BF16 |
| Seed | 42 |
Final train loss: 0.6852.
Research checkpoint for ARPO-style agentic SFT. Not evaluated as a standalone chat model. Follow the licenses of the base model and the SFT dataset.
@article{dong2025arpo,
author = {Guanting Dong and Hangyu Mao and Kai Ma and Licheng Bao and Yifei Chen and Zhongyuan Wang and Zhongxia Chen and Jiazhen Du and Huiyang Wang and Fuzheng Zhang and Guorui Zhou and Yutao Zhu and Ji-Rong Wen and Zhicheng Dou},
title = {Agentic Reinforced Policy Optimization},
journal = {CoRR},
volume = {abs/2507.19849},
year = {2025},
url = {https://arxiv.org/abs/2507.19849}
}