Downloads · 30 days
6
2% of all-time downloads
qwen-2.5-1.5b-rlvr-ppo
expx/qwen-2.5-1.5b-rlvr-ppo
qwen-2.5-1.5b-rlvr-ppo is a machine learning model from expx. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a fine-tuned version of ns-0/qwen-2.5-1.5b-instruct-reasoning-sft on the allenai/RLVR-GSM-MATH-IF-Mixed-Constraints dataset. It is specifically adapted to improve performance on mathematical and reasonin…
All-time downloads
257
Public
Parameters
1.5B
55.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 99%
At a glance
- Model type
- qwen2
- Access
- Public
- Created
- Jul 1, 2025
- Updated
- Jul 11, 2025
- SHA
- 25797f17
Base models
Model card
From the Hugging Face model README
This model is a fine-tuned version of ns-0/qwen-2.5-1.5b-instruct-reasoning-sft on the allenai/RLVR-GSM-MATH-IF-Mixed-Constraints dataset. It is specifically adapted to improve performance on mathematical and reasoning tasks as outlined by the dataset.
Model Details
- Base Model:
ns-0/qwen-2.5-1.5b-instruct-reasoning-sft - Dataset:
allenai/RLVR-GSM-MATH-IF-Mixed-Constraints - Training Method: Reinforcement Learning from Verifiable Rewards (RLVR) using a PPO (Proximal Policy Optimization) trainer.
- Framework: https://github.com/allenai/open-instruct/tree/main with Ray and vLLM.
Intermediate Checkpoints
mainbranch: This branch contains the final, fully trained model. (400K episodes)step-*branches: Each branch named step-* (e.g., step-100, step-200) contains an intermediate checkpoint saved at that specific training step. These are useful for research into model training dynamics etc.
Find logs here : https://wandb.ai/sauravpanigrahi-/open_instruct_internal
- Type
- qwen2
- Created
- Jul 1, 2025
- Updated
- Jul 11, 2025