Downloads · 30 days
0
yavuz-ai/qwen2.5-1.5b-grpo-gsm8k
qwen2.5-1.5b-grpo-gsm8k is a text generation model from yavuz-ai. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A LoRA policy from GRPO + RLVR on GSM8K (the DeepSeek-R1 / TinyZero recipe): the reward is verifiable ground truth (is the final number correct?), so there is no reward model and no value model. Part of grpo-gsm8k.
Downloads · 30 days
0
Access
Public
Updated Jun 19, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
Other1.5 KB · 52%
From the Hugging Face model README
A LoRA policy from GRPO + RLVR on GSM8K (the DeepSeek-R1 / TinyZero recipe): the reward is verifiable ground truth (is the final number correct?), so there is no reward model and no value model. Part of grpo-gsm8k.
GSM8K test accuracy 0.655 -> 0.665 (+1 pt, within noise at n=200); mean reasoning length 281 -> 291.
A small lift: the Instruct base is already strong on GSM8K, so a short run has little headroom
(R1-Zero-style "reasoning emerges" gains need a base model and much longer training). The value is the
method plus two findings - GRPO needs vLLM rollouts (~30x speedup over transformers.generate), and
headroom + run-length govern the gain. Full teardown in the repo.
GRPO (TRL, use_vllm colocate), LoRA r=16 / alpha=32, group size 8, beta 0, correctness + format
reward, 400 steps, 1x A100.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "yavuz-ai/qwen2.5-1.5b-grpo-gsm8k")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")