Downloads · 30 days
8
13% of all-time downloads
vera-gpt2-matryoshka
kofdai/vera-gpt2-matryoshka
vera-gpt2-matryoshka is a text generation model from kofdai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Personal-scale engineering recipe (not a new scientific principle): GPT-2 small weights fine-tuned under a frozen shared residual bottleneck (rank 256), with Matryoshka nested-rank training so one checkpoint works at…
All-time downloads
63
Public
Parameters
124M
501 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors498 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026
- SHA
- 305f5b35
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Vera GPT-2 Matryoshka (stereo-cross container)
Personal-scale engineering recipe (not a new scientific principle): GPT-2 small
weights fine-tuned under a frozen shared residual bottleneck (rank 256), with
Matryoshka nested-rank training so one checkpoint works at ranks
{8,16,32,64,128,192,256}. Prior art: Matryoshka Representation Learning + short KD.
Limits: GPT-2-class evidence only; scale invariance unverified. See
python -m vera about.
This is not stock GPT-2. Geometry (P, per-layer means) is applied at runtime by Vera hooks (or load via python -m vera).
Bundle layout
| File | Role |
|---|---|
model.safetensors | Fine-tuned GPT-2 weights |
vera_basis.npz | means [12,768], P [768,256] (and full V if present) |
config.json | Vera bundle metadata (kind=hook_container) |
Install / download
pip install transformers safetensors torch huggingface_hub
python -m vera convert --package-gpt2 # or: download suite
python -m vera chat --model hf:kofdai/vera-gpt2-matryoshka
Selecting GPT-2 in python -m vera ui downloads this repo together with the join partner kofdai/vera-distilgpt2-join.
Proven metrics (wikitext-2, research logs)
| Setting | Result |
|---|---|
| r=256 | ppl ≈ 77 (~1.41× vanilla GPT-2 baseline ~54.5) |
| Matryoshka | MATRYOSHKA_VIABLE — monotone degradation across ranks |
| Weights vs hooks | Weights trained; P/means frozen at inference |
Full chronicle: Ag3497120/Vera
How to load (Vera toolkit)
from vera.runtime import load_from_hf
vm = load_from_hf("kofdai/vera-gpt2-matryoshka", rank=256)
print(vm.generate("The capital of France is", max_new=20))
vm.set_rank(64) # Matryoshka lever
Citation / status
Research prototype from the Vera stereo-cross program (2026). Not a general chat replacement for modern LLMs — a reproducible container for shared coordinates, memory, and join experiments.
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026