OpenMath-Nemotron-1.5B-GRPO-NuminaMath-10K

OpenMath-Nemotron-1.5B-GRPO-NuminaMath-10K

GRPO (Group Relative Policy Optimization) LoRA fine-tune of nvidia/OpenMath-Nemotron-1.5B on the AI-MO/NuminaMath-CoT dataset (10K training samples).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("nvidia/OpenMath-Nemotron-1.5B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "ermiaazarkhalili/OpenMath-Nemotron-1.5B-GRPO-NuminaMath-10K")
tok = AutoTokenizer.from_pretrained("ermiaazarkhalili/OpenMath-Nemotron-1.5B-GRPO-NuminaMath-10K")

msgs = [{"role": "user", "content": "Solve: what is 17 * 24?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Intended use & limitations

Research / non-commercial experimentation on mathematical reasoning. As a LoRA adapter it inherits every limitation of the base model; verify outputs before use. A GGUF build (adapter merged onto the base) is available at ermiaazarkhalili/OpenMath-Nemotron-1.5B-GRPO-NuminaMath-10K-GGUF.