lattice-multiplication-transformer

Lattice Multiplication Transformer

Interactive Demo · Source Code

This repository contains the final checkpoint for an approximately 11.7-million-parameter model studying latent algorithm learning through an algorithm-aligned input representation.

Multi-digit multiplication is used as a case study. A learned frontend transforms two operands into an N × N lattice of digit-pair representations, which is passed to a standard Transformer encoder-decoder. The model directly generates the final product without scratchpads, carry labels, local-product labels, diagonal sums, or other intermediate supervision.

Results

The final checkpoint was trained for 600,000 optimization steps.

• 19,955 correct products across 20,000 square-length evaluation examples from 1 × 1 through 20 × 20
• 99.775% aggregate exact-match accuracy across those evaluations
• 100% exact-match accuracy from 1 × 1 through 15 × 15
• 96.5% exact-match accuracy and approximately 99.90% token accuracy at 20 × 20

Included Files

final_lattice_multiplication__11m.pt
lattice_multiplication_transformer.py
tokenizer.py
config.json

The checkpoint contains the model state, optimizer state, curriculum state, training history, and random-number-generator states. Inference requires only checkpoint["model_state_dict"].

Loading the Checkpoint

import json
import sys
from pathlib import Path

import torch
from huggingface_hub import snapshot_download


model_directory = Path(
    snapshot_download(
        repo_id="davidandroid/lattice-multiplication-transformer"
    )
)

sys.path.insert(0, str(model_directory))

from lattice_multiplication_transformer import (
    LatticeMultiplicationLoopTransformer,
)


with open(
    model_directory / "config.json",
    encoding="utf-8",
) as config_file:
    model_config = json.load(config_file)

model = LatticeMultiplicationLoopTransformer(
    **model_config
)

checkpoint = torch.load(
    model_directory / "final_lattice_multiplication__11m.pt",
    map_location="cpu",
    weights_only=False,
)

model.load_state_dict(
    checkpoint["model_state_dict"]
)

model.eval()

Limitations

The model was trained on operands containing at most 20 digits. It does not achieve reliable exact-match length generalization beyond that range, even though token accuracy remains higher immediately beyond the training boundary.

This is a research model for studying latent algorithm learning, not a replacement for deterministic multiplication software.