rukh-lora-e4

chorcat/rukh-lora-e4

A LoRA adapter for chorcat/rukh-medium. On its own it does nothing: it is 393,216 numbers (1.6 MB) that correct that model's weights, and without them there is no model to correct.

Written by hand rather than imported, as part of Rukh, a course that builds a chess language model end to end. How it works and why: https://lab.rukh.borjaglez.com.

What it is

The base weights never move. Everything outside A and B is frozen during training, which is what makes the file this small and what limits what it can learn: the correction has to pass through 8 dimensions per matrix.

What it changed

The model's own probability for e2e4 from the opening position, read off the softmax over the twenty legal first moves with no sampling -- so the number is a property of the weights and two readings agree to the last decimal:

And what the style cost, on the same suite every other stage of the project is measured with -- same validation positions, same puzzles, same seed:

How it was trained

200,000 games selected with split_part(uci, ' ', 1) = 'e2e4' AND white_elo >= 1800 AND black_elo >= 1800, from the 1800+ Lichess corpus of the project.

Using it

from rukh.models import MoveDecoder
from rukh.models.lora import load_adapter
from rukh.train import load_model

model, _ = load_model("<the base checkpoint>")
load_adapter(model, "adapter.safetensors")  # the config travels next to it

rukh.models.lora.merge_lora(model) folds it into the weights, after which the model is an ordinary MoveDecoder: same module names, same state dict, exportable to ONNX like any other.

In the browser

web/adapter.bin is the same factors again as one flat little-endian float32 buffer (1.6 MB): A first, then B, both stacked over the blocks in the order they run, with the alpha/r scaling already applied. web/adapter.json carries the shapes, because a buffer of floats says nothing about itself.

It is meant for the ONNX export of chorcat/rukh-medium that takes its LoRA factors as inputs of the graph (lora_a and lora_b) rather than baked into the weights. Fed an adapter of zeros that file is the base model exactly; fed this one it is this style. So the demo changes style by downloading 1.6 MB instead of a second copy of the model, which is the only reason a low-rank correction is worth keeping low-rank once it leaves the training loop.

In the course

• Built in M4 · Fine-tuning: los labs, the lesson that runs every command behind this repository.
• Measured in the single results table as the stage lora-e4: every stage of the course, the same suite, the same day.
• Bring it to the paths the configs read: uv run rukh pull lora-e4.

Licence

apache-2.0, the same as the base model.