Downloads · 30 days
18
3% of all-time downloads
gpt2-igc-is
jonfd/gpt2-igc-is
gpt2-igc-is is a text generation model from jonfd. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-4.0.
This Icelandic GPT-2 language model was pretrained on the Icelandic Gigaword Corpus (IGC, 2020 version), which contains approximately 1.532 million running words. The model was trained for 20 epochs on a TPU v3-8, wit…
All-time downloads
720
Public
Parameters
138M
1.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin513 MB · 34%
Parameter types
How the weights are stored.
F32125M · 91%
Datasets
Model card
From the Hugging Face model README
Icelandic GPT-2 model
This Icelandic GPT-2 language model was pretrained on the Icelandic Gigaword Corpus (IGC, 2020 version), which contains approximately 1.532 million running words. The model was trained for 20 epochs on a TPU v3-8, with a total training time of 3 days and 21 hours. The hyperparameters used for training can be found in the JAX/Flax documentation for the Transformers library. The model uses a byte-level BPE tokenizer with a vocabulary size of 51,000.
Note: This model was pretrained on a tokenized and sentence-segmentized version of the IGC, which is reflected by the generated text. A new version of this model, trained on a pre-tokenized version of IGC (2022 version), is forthcoming.
Acknowledgments
This research was supported with Cloud TPUs from Google's TPU Research Cloud (TRC).
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- cc-by-4.0
- Languages
- is
- Created
- Mar 30, 2023
- Updated
- Aug 29, 2023