Downloads · 30 days
4
12% of all-time downloads
qwen2.5-math-7b-baseline-grpo
resonancetech/qwen2.5-math-7b-baseline-grpo
qwen2.5-math-7b-baseline-grpo is a reinforcement learning model from resonancetech. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter trained with standard Group Relative Policy Optimization (GRPO) on GSM8K. This is the baseline model (no causal-aware reward) for comparison with the causal-aware variant.
All-time downloads
34
Public
Repo size
173 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors162 MB · 91%
At a glance
- Task
- Reinforcement Learning
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Mar 28, 2026
- Updated
- Mar 28, 2026
- SHA
- 891d1abc
Base models
Datasets
Model card
From the Hugging Face model README
Qwen2.5-Math-7B — Baseline GRPO (EXP-011)
LoRA adapter trained with standard Group Relative Policy Optimization (GRPO) on GSM8K. This is the baseline model (no causal-aware reward) for comparison with the causal-aware variant.
Key Results
| Metric | Causal (EXP-010) | Baseline (this model) |
|---|---|---|
| Mean reward | 0.394 | 0.255 |
| Peak reward | 0.560 | 0.400 |
Training Details
- Base model: Qwen/Qwen2.5-Math-7B-Instruct
- Method: GRPO (outcome + format reward only)
- Dataset: GSM8K (train split)
- Steps: 500
- Hardware: NVIDIA A40 (48GB)
- Precision: bf16, 4-bit QLoRA
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Math-7B-Instruct")
model = PeftModel.from_pretrained(base, "resonancetech/qwen2.5-math-7b-baseline-grpo")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Math-7B-Instruct")
Citation
Paper in preparation for NeurIPS 2026.
License
Apache 2.0 (following base model license)
- Task
- Reinforcement Learning
- Library
- peft
- License
- apache-2.0
- Languages
- en
- Created
- Mar 28, 2026
- Updated
- Mar 28, 2026