OMLCheT-v1

OMLCheT-v1

A fine-tuned distilgpt2 that generates legal chess moves in Standard Algebraic Notation (SAN), trained on ~86k real games from the Open Machine Learning Chess Tournament dataset.

---

Model Overview

The model treats a chess game as a text sequence: moves are space-separated tokens and the model is trained to predict the next token at each step. During inference, sampling from the model is equivalent to picking the next move.

---

Architecture Details

All figures are for the base distilgpt2 skeleton; the fine-tuning adds only one new embedding vector (<|chess|>).

---

Training Data

Training Porgress

What the corpus is and isn't:
The games come from ML-agent matches, not human grandmasters or large Lichess databases. This means the model has learned patterns produced by other (possibly imperfect) chess agents, not a broad human-style distribution. Move quality varies widely across the corpus.

---

Training Methodology

Supervised next-token prediction (standard causal language modelling). No reinforcement learning or RLHF was used.

Hyperparameters

Training process

1. Load distilgpt2 weights from HuggingFace Hub.
2. Add the <|chess|> domain prefix token and resize token embeddings.
3. Format each game: <|chess|> {san_moves} <EOS>.
4. Pack multiple short games per 256-token chunk to maximise GPU utilisation.
5. Train with cross-entropy loss over all tokens (moves and the prefix).
6. Select the checkpoint with the lowest eval_loss.

---

Known Limitations / Failure Modes

Tip for downstream users: always wrap inference in a legality filter (e.g. python-chess) and re-sample on illegal output.

---

Inference Speed

Benchmarked on a single NVIDIA T4 (Colab free tier) with the full fine-tuned checkpoint loaded in fp16:

On a Kaggle P100 expect roughly 2× faster; on CPU expect ~200–500 ms per move.

from transformers import pipeline
import torch

pipe = pipeline(
    "text-generation",
    model="OMLCheT/OMLCheT-v1",
    torch_dtype=torch.float16,
    device=0,                          # GPU; use -1 for CPU
)

# Provide moves played so far; model continues from here
prompt = "<|chess|> e4 e5 Nf3 Nc6 Bc4"
result = pipe(
    prompt,
    max_new_tokens=80,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    pad_token_id=pipe.tokenizer.eos_token_id,
)
print(result[0]["generated_text"])

---

License

MIT License

This model weights file is released under the MIT License.

• The base model (distilbert/distilgpt2) is also MIT-licensed.
• The training dataset (OMLCheT/chess-san-base) is released by us — check the dataset card for its specific terms.
• Chess move notation (SAN) is in the public domain.

You are free to use, modify, distribute, and build on top of this model for any purpose, commercial or non-commercial, with attribution.

---