Downloads · 30 days
34
19% of all-time downloads
mariklolik228/grpo-baseline-qwen2.5-1.5b-lora
grpo-baseline-qwen2.5-1.5b-lora is a machine learning model from mariklolik228. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter fine-tuned with standard GRPO (correctness + format reward only) for math reasoning on GSM8K. Baseline for the SuS method.
Downloads · 30 days
34
19% of all-time downloads
All-time downloads
178
Public
Repo size
307 MB
Likes
0
Public
Click a slice to open those files.
.safetensors295 MB · 95%
From the Hugging Face model README
LoRA adapter fine-tuned with standard GRPO (correctness + format reward only) for math reasoning on GSM8K. Baseline for the SuS method.
| Parameter | Value |
|---|---|
| LoRA rank (r) | 64 |
| LoRA alpha | 128 |
| Training steps | 2,000 |
| Learning rate | 5e-6 |
| Batch size | 8 |
| KL coefficient | 0.001 |
| Dataset | GSM8K |
| Metric | Score |
|---|---|
| Pass@1 | 73.98% |
| Pass@5 | 89.53% |
| Pass@8 | 91.88% |
95% CI (Pass@1): [72.10, 75.83]
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "mariklolik228/grpo-baseline-qwen2.5-1.5b-lora")
tokenizer = AutoTokenizer.from_pretrained("mariklolik228/grpo-baseline-qwen2.5-1.5b-lora")
@article{kashirskiy2026sus,
title={SuS: Strategy-aware Surprise for Intrinsic Exploration in GRPO},
author={Kashirskiy, Mark and Makarov, Ilya},
journal={arXiv preprint arXiv:2601.10349},
year={2026}
}