Downloads · 30 days
170
6% of all-time downloads
gpt-for-est-large
tartuNLP/gpt-for-est-large
gpt-for-est-large is a text generation model from tartuNLP. Use it when you need the model to write or continue text. It is set up for transformers.
This is the large-size GPT2 model, trained from scratch on 2.2 billion words (Estonian National Corpus + News Crawl + Common Crawl). Previously named "gpt-4-est-large", renamed to avoid click-baiting.
All-time downloads
2.9K
Public
Parameters
790M
12.3 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin3.1 GB · 50%
Parameter types
How the weights are stored.
F32765M · 97%
Try a prompt
Model card
From the Hugging Face model README
gpt-est-large
This is the large-size GPT2 model, trained from scratch on 2.2 billion words (Estonian National Corpus + News Crawl + Common Crawl). Previously named "gpt-4-est-large", renamed to avoid click-baiting.
Format
For training data was prepended with a text domain tag, and it should be added as prefix when using the model: >general<, >web<, >news<, >doaj< and >wiki< (standing for general texts, web crawled texts, news, article abstracts and wikipedia texts). Use the prefixes like this, e.g: ">web< Kas tead, et".
Model details
- num. of layers: 24
- num. of heads: 24
- embedding size: 1536
- context size: 1024
- total size: 723.58M params
Further details to be added soon.
Framework versions
- Transformers 4.13.0.dev0
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Created
- Mar 2, 2022
- Updated
- Nov 17, 2023