anlp-m26-a1-cipher-transformers

mdhvm/anlp-m26-a1-cipher-transformers

Checkpoints for Advanced NLP Assignment 1 — An encoder–decoder Transformer, written from elementary PyTorch operations, is trained to recover plaintext from a repeating-key XOR cipher. Five configurations differ from the base model in exactly one component each — rotary positions, grouped-query attention, RMSNorm, and a token-free Byte Latent Transformer with entropy-based dynamic patching — and are compared at an identical step budget over three seeds.

Loading

import torch, sys
sys.path.insert(0, "src")
from train import CONFIGS, build_model

ck = torch.load("C1/best.pt", map_location="cpu")
# build_model(CONFIGS["C1"], ck["hp"], bundle, device) -- see src/evaluate.py

Training code, tests and the full study: see the accompanying submission.

Files