Downloads · 30 days
12
12% of all-time downloads
Hooshaai/Qwen3-4B-GSM8k-GRPO-Unsloth
Qwen3-4B-GSM8k-GRPO-Unsloth is a machine learning model from Hooshaai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a Qwen/Qwen3-4B model fine-tuned on a 10% subset of the GSM8k dataset using GRPO (Grounded Reward Preference Optimization). The training was accelerated using the Unsloth library.
Downloads · 30 days
12
12% of all-time downloads
All-time downloads
98
Public
Parameters
4B
3.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.6 GB · 100%
How the weights are stored.
U83B · 75%
From the Hugging Face model README
This is a Qwen/Qwen3-4B model fine-tuned on a 10% subset of the GSM8k dataset using GRPO (Grounded Reward Preference Optimization). The training was accelerated using the Unsloth library.
The model was trained to solve grade-school math problems by following a strict XML-like format for reasoning and providing a final answer.
Qwen/Qwen3-4Btrl.GRPOTrainer)openai/gsm8k.This is a LoRA adapter and must be loaded on top of the base model using Unsloth for the best performance and compatibility.
import torch
from unsloth import FastLanguageModel
# Load the base model with Unsloth, using the same settings as training
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="Qwen/Qwen3-4B",
max_seq_length=2048,
load_in_4bit=True,
dtype=torch.bfloat16, # Use bfloat16 for inference
)
# Load the LoRA adapter from the Hub
model = FastLanguageModel.from_pretrained(
model=model,
model_name="tahamajs/Qwen3-4B-GSM8k-GRPO-Unsloth", # Your Hub model ID
)
# --- Define Prompt and Generate ---
SYSTEM_PROMPT = (
"You are a helpful assistant.\n"
"First think through the problem, then provide the answer.\n"
"Use this strict format:\n"
"<reasoning>\n"
"your step-by-step reasoning here\n"
"</reasoning>\n"
"<answer>\n"
"The final answer is [final_number].\n"
"</answer>\n"
)
question = "Natalia sold 48 liters of milk in the morning. In the afternoon, she sold 27 liters less than in the morning. In the evening, she sold 15 liters more than in the afternoon. How many liters of milk did she sell in total?"
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": question},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True)
response = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
# Extract only the generated part
generated_part = response.split("<|im_start|>assistant\n")[-1]
print(generated_part)
The model was trained using trl's GRPOTrainer.
r_correctness) with a weight of 2.0.β): 0.01r): 160.00059k): 3