Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K

Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K

GRPO (Group Relative Policy Optimization) LoRA fine-tune of meta-llama/Llama-3.2-3B-Instruct on the AI-MO/NuminaMath-CoT dataset (10K training samples).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K")
tok = AutoTokenizer.from_pretrained("ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K")

msgs = [{"role": "user", "content": "Solve: what is 17 * 24?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Intended use & limitations

Research / non-commercial experimentation on mathematical reasoning. As a LoRA adapter it inherits every limitation of the base model; verify outputs before use. A GGUF build (adapter merged onto the base) is available at ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K-GGUF.