Downloads · 30 days
42
2% of all-time downloads
gpt2-wechsel-german-ds-meg
malteos/gpt2-wechsel-german-ds-meg
gpt2-wechsel-german-ds-meg is a text generation model from malteos. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
- trained with BigScience's DeepSpeed-Megatron-LM code base - 22hrs on 4xA100 GPUs (~ 80 TFLOPs / GPU) - stopped after 100k steps - less than a single epoch on oscarunshuffleddeduplicatedde (excluding validation set;…
All-time downloads
2.7K
Public
Parameters
137M
1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin511 MB · 50%
Parameter types
How the weights are stored.
F32124M · 91%
Try a prompt
Model card
From the Hugging Face model README
Replication of gpt2-wechsel-german
- trained with BigScience's DeepSpeed-Megatron-LM code base
- 22hrs on 4xA100 GPUs (~ 80 TFLOPs / GPU)
- stopped after 100k steps
- less than a single epoch on
oscar_unshuffled_deduplicated_de(excluding validation set; original model was trained for 75 epochs on less data) - bf16
- zero stage 1
- tp/pp = 1
Evaluation
| Model | PPL |
|---|---|
gpt2-wechsel-german-ds-meg | 26.4 |
gpt2-wechsel-german | 26.8 |
gpt2 (retrained from scratch) | 27.63 |
License
MIT
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- de
- Created
- May 5, 2022
- Updated
- Mar 24, 2023