Downloads · 30 days
34
17% of all-time downloads
mariklolik228/sus-qwen2.5-1.5b-grpo-lora
sus-qwen2.5-1.5b-grpo-lora is a machine learning model from mariklolik228. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter fine-tuned with SuS (Strategy-aware Surprise) + GRPO for math reasoning on GSM8K.
Downloads · 30 days
34
17% of all-time downloads
All-time downloads
203
Public
Repo size
307 MB
Likes
0
Public
Click a slice to open those files.
.safetensors295 MB · 95%
From the Hugging Face model README
LoRA adapter fine-tuned with SuS (Strategy-aware Surprise) + GRPO for math reasoning on GSM8K.
SuS adds a semantic novelty bonus to correct responses in mixed-correctness batches, encouraging diverse problem-solving strategies. Zero-variance batches (all-correct or all-incorrect) are left untouched — this prevents KL divergence blowup and length collapse.
| Parameter | Value |
|---|---|
| LoRA rank (r) | 64 |
| LoRA alpha | 128 |
| Training steps | 2,000 |
| Learning rate | 5e-6 |
| Batch size | 8 |
| KL coefficient | 0.001 |
| SS bonus (β) | 0.1 |
| Dataset | GSM8K |
| Metric | Score |
|---|---|
| Pass@1 | 75.27% |
| Pass@5 | 94.03% |
| Pass@8 | 97.63% |
95% CI (Pass@1): [73.49, 77.05]
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "mariklolik228/sus-qwen2.5-1.5b-grpo-lora")
tokenizer = AutoTokenizer.from_pretrained("mariklolik228/sus-qwen2.5-1.5b-grpo-lora")
@article{kashirskiy2026sus,
title={SuS: Strategy-aware Surprise for Intrinsic Exploration in GRPO},
author={Kashirskiy, Mark and Makarov, Ilya},
journal={arXiv preprint arXiv:2601.10349},
year={2026}
}