Reversi-Transformer-1

Transformer Reversi AI

This Reversi AI is a Transformer-based model that uses Mixture-of-Experts architecture for decision-making instead of CNNs.
Github - rsu-Suba/ReversiGPT

Architecture

• Parameters: 1.68M
• Model size: 20.6MB
• Input: (8, 8, 2)
  • Own stones
  • Opponent stones
• Board representation: 64 tokens
• Embedding dimension: 128
• Transformer blocks: 4
• Attention heads: 4
• MHA experts: 2
• FFN experts: 2
• Routing: softmax-based dynamic routing
• Iterative routing steps: 2
• Normalization: Pre-LN LayerNorm
• FFN activation: GELU

Diagram

Input (8×8×2)
    ↓
64 Tokens
    ↓
Dense 128
    ↓
Token + Row + Column + Move Embeddings
    ↓
DynamicAssembly × 4
  ├─ MHA Expert × 2
  └─ FFN Expert × 2
    ↓
    ├─ Policy Head → 64 moves
    └─ Value Head → Win-rate

DynamicAssembly

Each DynamicAssembly block contains a shared pool of MHA and FFN experts. At each iteration step, the mean-pooled board representation is combined with a step embedding and passed to a router.

The router produces a probability distribution over the expert pool. All experts are evaluated and their outputs are combined using the router probabilities.

Policy Head

The policy head predicts a probability distribution over the 64 board positions.

Value Head

The value head predicts a scalar value in [-1, 1], representing the estimated game outcome.

Training

This model was trained using self-play game records generated by a previous CNN-based Othello AI.

Self-Play data generation

• MCTS simulations: 600
• MCTS batch size: 100
• PUCT constant: 2.3
• Total games: 10,000

Model Training

• Optimizer: AdamW
• Clipnorm: 1.0
• Weight decay: 0.05
• Learning rate: 6.0e-5
• Learning rate scheduler: ReduceLROnPlateau
• Training batch size: 256
• Epochs: 75

Evaluation

A 63M-parameter ResNet CNN model was used as the previous model.

Win rate

• vs Random : 100% (100 / 100 games)
• vs Prev model : 63% (63 / 100 games)

Average final stone count

• vs Random : 58
• vs Prev model : 47

Usage / Quick start

import numpy as np
import tensorflow as tf 
from huggingface_hub import hf_hub_download
from model import TokenAndPositionEmbedding, MHA, FFN, DynamicAssembly

model_path = hf_hub_download(repo_id="rsu/Reversi-Transformer-1", filename="M1.h5", repo_type="model")

custom_objects = {
  "TokenAndPositionEmbedding": TokenAndPositionEmbedding,
  "MHA": MHA,
  "FFN": FFN,
  "DynamicAssembly": DynamicAssembly,
}
model = tf.keras.models.load_model(model_path, custom_objects=custom_objects, compile=False)

board = np.zeros((1, 8, 8, 2), dtype=np.float32)
board[0, 3, 3, 1] = 1.0
board[0, 3, 4, 1] = 1.0
board[0, 3, 3, 1] = 1.0
board[0, 4, 4, 1] = 1.0

policy, value = model(board, training=False)
print("Policy (64 moves prob): ", policy.numpy())
print("Value (win rate est [-1, 1]): ", value.numpy().item())