Llama-2-7B-ProXMath

Llama-2-7B-ProXMath

<p align="center"> <img src="prox-teaser.png"> </p>

ArXiv | Data: OpenWebMath-Pro | Code

Llama-2-7B-ProXMath is a math-adapted Llama-2-7B model that is continually pre-trained on OpenWebMath-Pro (a refined version by ProX) for 10B tokens.

Evaluations

ProX models are evaluated on 9 common math reasoning benchmarks.

Citation

@article{zhou2024programming,
  title={Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale},
  author={Zhou, Fan and Wang, Zengzhi and Liu, Qian and Li, Junlong and Liu, Pengfei},
  journal={arXiv preprint arXiv:2409.17115},
  year={2024}
}