Qwen2.5-Math-1.5B-RoPE-300k

Qwen2.5-Math-1.5B-RoPE-300k

This model is a variant of Qwen/Qwen2.5-Math-1.5B, whose RoPE base frequency was increased to 300k in order to extend the model's context from 4k to 32k tokens. The weights are unchanged; only config.json differs (rope_theta 10000 → 300000, max_position_embeddings 4096 → 32768, sliding_window cleared). It is the 1.5B analogue of open-r1/Qwen2.5-Math-7B-RoPE-300k, intended as a base for long-context reasoning SFT.

Citation

@article{yang2024qwen25mathtechnicalreportmathematical,
  title={Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement}, 
  author={An Yang and Beichen Zhang and Binyuan Hui and Bofei Gao and Bowen Yu and Chengpeng Li and Dayiheng Liu and Jianhong Tu and Jingren Zhou and Junyang Lin and Keming Lu and Mingfeng Xue and Runji Lin and Tianyu Liu and Xingzhang Ren and Zhenru Zhang},
  journal={arXiv preprint arXiv:2409.12122},
  year={2024}
}