Downloads · 30 days
0
satyam-arora-iiit-hyderabad/anlp-m26-a1-checkpoints
anlp-m26-a1-checkpoints is a machine learning model from satyam-arora-iiit-hyderabad. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains five from-scratch PyTorch Transformer configurations for encrypted-binary-sequence to plaintext reconstruction.
Downloads · 30 days
0
Access
Public
Updated Sep 2, 2026
Repo size
327 MB
Likes
0
Public
Click a slice to open those files.
.pt163 MB · 96%
From the Hugging Face model README
This repository contains five from-scratch PyTorch Transformer configurations for encrypted-binary-sequence to plaintext reconstruction.
C1-C4 use learned custom BPE tokenizers implemented without a tokenizer library. C5 uses packed ciphertext bytes and dynamically sized patches derived from next-byte entropy.
| Configuration | Position | Attention | Normalization | Representation |
|---|---|---|---|---|
| C1 | Sinusoidal | MHA | LayerNorm | Custom BPE |
| C2 | RoPE | MHA | LayerNorm | Custom BPE |
| C3 | Sinusoidal | GQA, 8 query/2 KV heads | LayerNorm | Custom BPE |
| C4 | Sinusoidal | MHA | RMSNorm | Custom BPE |
| C5 | Sinusoidal | MHA | LayerNorm | Packed bytes + dynamic entropy patches |
C2-C4 each change one architectural component relative to C1. C5 replaces the subword interface with a simplified token-free BLT pipeline.
All results use greedy decoding on the same 1,389-example test split.
| Config | Bit accuracy | Sequence accuracy | Mean Levenshtein | BLEU | ROUGE-L |
|---|---|---|---|---|---|
| C1 | 0.874859 | 0.293017 | 3.392369 | 90.369524 | 0.955734 |
| C2 | 0.949647 | 0.598272 | 2.472282 | 95.937859 | 0.981416 |
| C3 | 0.814852 | 0.120230 | 8.535637 | 80.490798 | 0.910088 |
| C4 | 0.854665 | 0.226062 | 4.676746 | 87.776615 | 0.944754 |
| C5 | 0.979374 | 0.231821 | 6.260619 | N/A | N/A |
BLEU and ROUGE are reported only for tokenized C1-C4. Cross-entropy loss is not directly compared between subword and byte prediction units.
| Config | Parameters | Mean epoch (s) | Maximum recorded memory (GiB) |
|---|---|---|---|
| C1 | 8,530,896 | 54.885 | 1.376 |
| C2 | 8,530,896 | 60.448 | 1.371 |
| C3 | 7,346,640 | 54.151 | 1.356 |
| C4 | 8,525,264 | 50.055 | 1.292 |
| C5 | 7,709,699 | 89.552 | 1.306 |
| Measure | C5 delta |
|---|---|
| Mean complete epoch | +63.16% |
| Mean training step | +106.62% |
| Training-step peak allocated memory | +24.75% |
| Parameters | -9.63% |
These are custom educational PyTorch architectures and are not compatible
with Hugging Face AutoModel.
Earlier prebuilt-tokenizer and fixed-patch runs are retained only as a superseded audit trail.