Downloads · 30 days
0
anlp-a1-transformers
mohjkhan/anlp-a1-transformers
anlp-a1-transformers is a machine learning model from mohjkhan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
Custom encoder–decoder Transformer built from fundamental PyTorch ops (no nn.Transformer / nn.MultiheadAttention) for a repeating-XOR cipher → plaintext decryption task, plus an ablation over five configurations:
Access
Public
Updated Aug 30, 2026
Repo size
236 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt236 MB · 100%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Aug 30, 2026
- SHA
- c8e39f57
Model card
From the Hugging Face model README
ANLP Assignment 1 — Transformers from Scratch (C1–C5)
Custom encoder–decoder Transformer built from fundamental PyTorch ops (no
nn.Transformer / nn.MultiheadAttention) for a repeating-XOR cipher →
plaintext decryption task, plus an ablation over five configurations:
| Config | Positional | Attention | Norm | Tokenization |
|---|---|---|---|---|
| C1 | Sinusoidal | Multi-Head | LayerNorm | Subword |
| C2 | RoPE | Multi-Head | LayerNorm | Subword |
| C3 | Sinusoidal | Grouped-Query | LayerNorm | Subword |
| C4 | Sinusoidal | Multi-Head | RMSNorm | Subword |
| C5 | Sinusoidal | Multi-Head | LayerNorm | BLT (token-free) |
Each *.pt file contains {"model": state_dict, "config": {...}}. Load with the
build_transformer / build_blt factories in the accompanying code repository.
- Training logs (Weights & Biases): <ADD-WANDB-LINK>
- Code: <ADD-CODE-REPO-LINK>
Metrics for each configuration are in the corresponding *_metrics.json.
- Library
- pytorch
- License
- mit
- Created
- Aug 29, 2026
- Updated
- Aug 30, 2026