Downloads · 30 days
8
13% of all-time downloads
vera-distilgpt2-join
kofdai/vera-distilgpt2-join
vera-distilgpt2-join is a text generation model from kofdai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Second model of the GPT-2 join suite: DistilGPT2 (6 layers, 768-d) distilled onto the same frozen shared basis P as kofdai/vera-gpt2-matryoshka.
All-time downloads
60
Public
Parameters
81.9M
331 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors328 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026
- SHA
- bf4ec5de
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Vera DistilGPT2 Join Partner (shared stereo-cross P)
Second model of the GPT-2 join suite: DistilGPT2 (6 layers, 768-d) distilled onto the same frozen shared basis P as kofdai/vera-gpt2-matryoshka.
Honest framing: prior-art model-stitching probe at personal scale — not a universal coordinate dictionary. P is reused from model A (reuse bias). Join ≫ random/shuffle is a weak lineage / shared-rep signal (python -m vera audit). Scale-up unverified (python -m vera about).
Used for cross-model coordinate join demos (Step 5 JOIN_VIABLE at GPT-2 class): mid-layer coords from GPT-2 → DistilGPT2 stay nearer solo than random/shuffle controls.
Bundle layout
| File | Role |
|---|---|
model.safetensors | Fine-tuned DistilGPT2 weights |
vera_basis.npz | DistilGPT2 per-layer means + shared P (same dictionary as GPT-2 Matryoshka) |
config.json | kind=hook_container, role=join_partner |
Download with the GPT-2 suite
python -m vera download-gpt2-suite
# pulls:
# kofdai/vera-gpt2-matryoshka
# kofdai/vera-distilgpt2-join
Or select GPT-2 (Vera suite) in python -m vera ui → Convert/Chat — both repos are fetched.
Proven join snapshot
| Direction | Site | Real join vs solo | Random control |
|---|---|---|---|
| GPT-2 → DistilGPT2 | late | ~1.01× solo ppl | tens–hundreds× worse |
| DistilGPT2 → GPT-2 | early | ~1.07× | collapse |
Details: CROSS_MODEL_JOIN.md in the Vera repo.
Load
from vera.runtime import load_from_hf
partner = load_from_hf("kofdai/vera-distilgpt2-join", rank=256)
Must be paired with the Matryoshka GPT-2 bundle that shares the same P columns.
Status
Research partner checkpoint for stereo-cross join — not a standalone chat model.
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026