Downloads · 30 days
18
1% of all-time downloads
gpt2-wiki103
irodkin/gpt2-wiki103
gpt2-wiki103 is a text generation model from irodkin. Use it when you need the model to write or continue text. It is set up for transformers.
(!) Don't forget to preprocess unknowntokens and substitute them with <|endoftext|. Otherwise the \<unk\ tokens in dataset will be split into the '<', 'unk' and '' tokens
All-time downloads
1.9K
Public
Parameters
124M
2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin498 MB · 50%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- Dec 29, 2023
- Updated
- Nov 7, 2024
- SHA
- 03e952a7
Try a prompt
Datasets
Model card
From the Hugging Face model README
(!) Don't forget to preprocess unknown_tokens and substitute them with <|endoftext|>. Otherwise the <unk> tokens in dataset will be split into the '<', 'unk' and '>' tokens
- Full context (1024) perplexity on test set: 13.68
Dependence of the cross entropy loss on the length of the context for prediction
- x-axis*128 = context length
- y-axis = cross entropy
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Languages
- en
- Created
- Dec 29, 2023
- Updated
- Nov 7, 2024