Downloads · 30 days
28
34% of all-time downloads
gpt-2-wikitext-chunks-epsroot-0-bs64
EleutherAI/gpt-2-wikitext-chunks-epsroot-0-bs64
gpt-2-wikitext-chunks-epsroot-0-bs64 is a machine learning model from EleutherAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
GPT-2 (124M) fine-tuned on EleutherAI/bergson-wikitext-512-chunks with the bergson MAGIC trainer, for use as a base model in training data attribution experiments.
All-time downloads
82
Public
Parameters
124M
498 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors498 MB · 99%
At a glance
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026
- SHA
- 5ace30d5
Base models
Model card
From the Hugging Face model README
gpt-2-wikitext-chunks-epsroot-0-bs64
GPT-2 (124M) fine-tuned on EleutherAI/bergson-wikitext-512-chunks with the bergson MAGIC trainer, for use as a base model in training data attribution experiments.
Training details
- Optimizer: AdamW (beta1=0.95, beta2=0.975,
eps_root=0, weight_decay=0.01) - Batch size: 64, epochs: 4, seed: 42, precision: fp32
- LR: polynomial schedule, peak 8e-4, warmup 25%, end 8e-5
- Trained with bergson 0.10.1 (git sha
27eb22b3)
Replication
The exact bergson run config is included as bergson_config.yaml. Replay it with:
bergson bergson_config.yaml
A companion model trained identically but with eps_root=1e-6 is available at EleutherAI/gpt-2-wikitext-chunks-epsroot-1e-6-bs64.
- Type
- gpt2
- License
- mit
- Languages
- en
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026