Qwen-2.5-3B-Instruct-TicTacToe-GRPO

Tic-Tac-Toe Reinforcement Learning Model

This model was trained to play Tic-Tac-Toe using reinforcement learning through ART (AI Reinforcement Training).

Model Details

• Base Model: Qwen/Qwen2.5-3B-Instruct
• Training Method: Reinforcement Learning with Human Feedback (RLHF)
• Task: Playing Tic-Tac-Toe optimally

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("banachspace/Qwen-2.5-3B-Instruct-TicTacToe-GRPO")
tokenizer = AutoTokenizer.from_pretrained("banachspace/Qwen-2.5-3B-Instruct-TicTacToe-GRPO")

# Example usage for Tic-Tac-Toe
messages = [
    {"role": "system", "content": "You are a tic-tac-toe player. Return your move as an XML object like <move>A1</move>. You are the x symbol."},
    {"role": "user", "content": "   1   2   3\nA  _ | _ | _\nB  _ | _ | _\nC  _ | _ | _"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=50)
response = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
print(f"AI move: {response}")

Training Process

The model was trained using generative reinforcement learning on the task of playing Tic-Tac-Toe optimally.

• Training framework: ART (AI Reinforcement Training)
• Reward function: Win = 1.0, Draw = 0.5, Loss = 0.0
• Learning method: Proximal Policy Optimization (PPO)