gemma-3-1b-cure-grpo

Model Card for gemma-3-1b-cure-grpo

This model is a fine-tuned version of unsloth/gemma-3-1b-it. It has been trained using TRL.

Quick start

from transformers import pipeline

# 1. Define the System Prompt that triggers the CURE behavior
SYSTEM_PROMPT = """You are an expert mathematician. You must solve the user's problem using the strict CURE format:
1. [initial_reasoning]: Draft the solution.
2. [critique]: Check for errors in the draft.
3. [refined_reasoning]: Fix errors and write the correct steps.
4. [uncertainty]: Estimate confidence (0.0-1.0).
5. [verify]: Check the final logic.
6. [final_reasoning]: Summarize the solution.

Output the final numeric answer inside <answer> tags.
DO NOT ask the user for more information. Solve it with the data given."""

# 2. Load the model
generator = pipeline("text-generation", model="RinKana/gemma-3-1b-cure-grpo", device="cuda")

# 3. Ask a reasoning question
question = "James buys 5 packs of beef that are 4 pounds each. The price of beef is $5.50 per pound. How much did he pay?"

messages = [
    {"role": "user", "content": SYSTEM_PROMPT + "\n\nQuestion:\n" + question}
]

# 4. Generate response
output = generator(messages, max_new_tokens=768, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with GRPO, a method introduced in DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models and Unsloth on a single Tesla T4 GPU. It utilizes the GRPO algorithm to optimize for format adherence, correctness, and conciseness without needing a separate reference model (saving VRAM).

• Method: GRPO (Generalized Reasoning Preference Optimization)
• Base Model: unsloth/gemma-3-1b-it
• Optimization Goals: Format compliance, self-correction, and brevity.
• Training Steps: 150
• Result(on 50 test-set GSM8K): 48% (24/50)

Framework versions

• TRL: 0.22.2
• Transformers: 4.56.2
• Pytorch: 2.7.0
• Datasets: 4.3.0
• Tokenizers: 0.22.1

Citations

Cite GRPO as:

@article{shao2024deepseekmath,
    title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
    author       = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
    year         = 2024,
    eprint       = {arXiv:2402.03300},
}

Cite TRL as:

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}