Downloads · 30 days
176
8% of all-time downloads
GPT-2-wikitext-chunks
EleutherAI/GPT-2-wikitext-chunks
GPT-2-wikitext-chunks is a text generation model from EleutherAI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
GPT-2 (124M) fine-tuned on EleutherAI/bergson-wikitext-512-chunks (WikiText-2 pre-chunked to 512-token sequences, 4,608 train chunks) using the bergson MAGIC trainer, as the trained model for MAGIC attribution experim…
All-time downloads
2.1K
Public
Parameters
124M
996 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt498 MB · 50%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Jul 8, 2026
- Updated
- Jul 8, 2026
- SHA
- 8bbd516d
Try a prompt
Base models
Model card
From the Hugging Face model README
GPT-2 fine-tuned on bergson-wikitext-512-chunks
GPT-2 (124M) fine-tuned on EleutherAI/bergson-wikitext-512-chunks (WikiText-2 pre-chunked to 512-token sequences, 4,608 train chunks) using the bergson MAGIC trainer, as the trained model for MAGIC attribution experiments.
Training
- 4 epochs, global batch size 64 (8x data parallel), 288 steps
- AdamW, polynomial LR schedule: lr 8e-4 (start 1e-6, end 8e-5), 25% warmup, fp32
- Loss on held-out
test[:4]chunks: 3.22 (base gpt2: 3.62)
Files
- Standard HF model + tokenizer files
bergson_config.yaml— the fully-resolved bergson run config (all fields incl. defaults) that produced this model; rerun withpython -m bergson bergson_config.yamloptimizer.pt— AdamW second moments (exp_avg_sq) at the final training step, in bergson'soptimizer.ptnormalizer format ({"state": {idx: {"exp_avg_sq": ...}}, "param_groups": [...]}withidxindexing deduplicatedmodel.named_parameters()), for gradient normalization in attribution runs.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Created
- Jul 8, 2026
- Updated
- Jul 8, 2026