Downloads · 30 days
0
unpairedelectron07/ChessGRPO-The-Weight-of-Silence
ChessGRPO-The-Weight-of-Silence is a text generation model from unpairedelectron07. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapters accompanying "The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning." Paper (arXiv) · Code (GitHub) · Zenodo · Training Data (Kaggle)
Downloads · 30 days
0
Access
Public
Updated Jul 28, 2026
Repo size
1.6 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.5 GB · 96%
From the Hugging Face model README
LoRA adapters accompanying "The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning." Paper (arXiv) · Code (GitHub) · Zenodo · Training Data (Kaggle)
This repository contains six LoRA adapters, all fine-tuned from Qwen/Qwen3-14B, tracing a chess-playing model through supervised fine-tuning, two explicit chain-of-thought reinforcement learning runs, a latent-reasoning curriculum, and two reinforcement learning runs on top of that latent curriculum. Together they form the full experimental arc behind the paper's central finding: reinforcement learning improves legal-move rate and eliminates checkmate confabulation, but causal intervention shows the model does not depend on the content of its latent "thoughts" to achieve either gain. The improvement is encoded in the adapter weights, not in inference-time computation over the thought vectors.
| Subfolder | Stage | Legal move rate | Accuracy | False checkmates | Notes |
|---|---|---|---|---|---|
sft-qwen3-14b-chess | Imitation baseline | 38% | 9% | 28 | Supervised fine-tuning only, no reinforcement learning. |
rung1-grpo-v1-explicit-chess | Explicit CoT + RL (sparse reward) | 52% | 8% | 0 | Superseded. Sparse reward let legality crowd out move quality; Stockfish centipawn-loss comparison shows degraded play relative to the SFT baseline despite the legal-move gain. Retained for provenance. |
rung1-grpo-v2-explicit-chess | Explicit CoT + RL (gated dense reward) | 52% | 10% | 0 | Fixes the v1 quality regression with a gated dense reward. This is the explicit-reasoning result reported in the paper. |
rung2-stage2-latent-chess | Latent curriculum, no RL | 48% | 10% | 19 | The model's reasoning has been compressed into latent thought vectors, but no reinforcement learning has been applied yet. |
rung3-latent-grpo-chess | Latent curriculum + RL | 61% | 9% | 0 | The headline result. Highest legal-move rate across every checkpoint, with false checkmates fully eliminated. |
rung3-gumbel-chess | Latent curriculum + RL (Gumbel-Softmax variant) | ~50% | — | 0 | A second, independent RL run using Gumbel-Softmax straight-through estimation instead of the primary Rung 3 method. Confirms the same qualitative pattern with a smaller absolute gain; included as a robustness check, not a replacement for rung3-latent-grpo-chess. |
Format compliance is 100% for every checkpoint except where the causal interventions described in the paper deliberately push the model out of distribution.
Every checkpoint is a LoRA adapter, not a standalone model. Loading one requires the base model and the peft library. Select a checkpoint using subfolder.
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch
BASE_MODEL = "unsloth/qwen3-14b-bnb-4bit" # 4-bit NF4 base used during training and evaluation
REPO_ID = "unpairedelectron07/ChessGRPO-The-Weight-of-Silence"
CHECKPOINT = "rung3-latent-grpo-chess" # swap for any subfolder in the table above
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
)
base = AutoModelForCausalLM.from_pretrained(
BASE_MODEL, quantization_config=bnb_config, dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, subfolder=CHECKPOINT)
model = PeftModel.from_pretrained(base, REPO_ID, subfolder=CHECKPOINT)
model.eval()
The explicit-reasoning checkpoints (sft-qwen3-14b-chess, rung1-grpo-v1-explicit-chess, rung1-grpo-v2-explicit-chess) expect written reasoning inside <thinking> tags, structured under [KING SAFETY], [CHECKS], [CAPTURES & TRADES], [THREATS], and [IMPROVEMENT] headers, followed by a move in UCI format inside <output> tags.
The latent checkpoints (rung2-stage2-latent-chess, rung3-latent-grpo-chess, rung3-gumbel-chess) replace written reasoning with a fixed number of continuous "thought" vectors generated between two special tokens, rather than text. Reproducing this behavior requires the thought-generation loop described in the paper and implemented in the GitHub repository's interpretability/ and rung3_latent_grpo/ scripts; a chat template alone is not sufficient for these checkpoints.
Reading the model card alone might suggest the latent checkpoints "think silently" in a way analogous to explicit chain-of-thought. The paper's central finding is that this is not the case. A six-condition causal intervention suite, applied to both rung2-stage2-latent-chess and rung3-latent-grpo-chess, shows that substituting, adding noise to, or entirely ablating the latent thought vectors leaves performance largely unchanged. Only replacing them with all-zero vectors causes a collapse, which reflects an out-of-distribution failure rather than evidence that the thought content was doing meaningful work. Full methodology and statistical testing are in the paper.
Training data (12,002 rows of FEN positions, best moves, and structured reasoning) is available separately on Kaggle. Link forthcoming.
These adapters are released under the Apache 2.0 license, matching the base model. Use of these adapters requires compliance with the Qwen3-14B license terms for the base model itself.
@misc{kshirsagar2026weightofsilence,
title = {The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning},
author = {Kshirsagar, Ishan S.},
year = {2026},
eprint = {2607.20952},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2607.20952}
}
Ishan S. Kshirsagar, independent researcher. Paper: arxiv.org/abs/2607.20952