llama3-8b-sft-grpo

Llama-3-8b-sft-grpo

The model was trained for the LM Playschool Challenge (beta).
It is designed to play games in ClemBench.

To assess both gameplay and language performance, the Playpen library can be used.

Model description

• Model type: A model trained on publicly available clembench data.
• Language(s) (NLP): Primarily English
• License: Llama 3.1 Community License Agreement
• Finetuned from model: pm-25/llama3-8b-sft

Model Sources

• Training Repository: https://github.com/paulutsch/playpen
• Eval Repository: https://github.com/lm-playpen/playpen

Training Data

The model was trained on a processed and filtered version of the clembench DPO Turn dataset, using additionally created scoring functions for automatically verifiable rewards

Specifically, we used:

• clembench-dpo-turn

Model Family

Using the model

Loading with HuggingFace

To load the model with HuggingFace, use the following snippet:

from transformers import AutoModelForCausalLM

grpo_model = AutoModelForCausalLM.from_pretrained("pm-25/llama3-8b-sft-grpo")

via Playpen

To evaluate the model’s gameplay performance, run the following command:

playpen eval <model-name>

Before evaluation, the model must be registered in the model_registry.json file located in the playpen folder:

{
"model_name": "llama3-8b-sft-grpo",
"backend": "huggingface_local",
"huggingface_id": "pm-25/llama3-8b-sft-grpo",
"release_date": "2025-08-22",
"open_weight": true,
"parameters": "8B",
"languages": ["en", "de", "fr", "it", "pt", "hi", "es", "th"],
"context_size": "128k",
"license": {
"name": "Meta",
"url": "https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE"
},
"model_config": {
"requires_api_key": true,
"premade_chat_template": true,
"eos_to_cull": "<\|eot_id\|>"
}
}

Performance

Hyperparameters

GRPO:

• Learning Rate: 5e-6
• Effective Batch Size: 16
• Max. Sequence Length: 4096
• Loss Accumulation: Sum
• Learning Rate Schedule: Linear
• LR Warmup Ratio: 0.03
• Num. Epochs: 2
• bf16: True
• Seed: 7331

LoRA Config:

• r: 16
• lora_alpha: 32
• lora_dropout: 0.05
• Target Modules: All Linear
• Modules to Save: lm_head, embed_tokens

License and use

All Llama 3.1 models are released under Meta's Llama 3.1 Community License Agreement. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. It is intended for research and educational use. For more information, please see our Responsible Use Guidelines.