Downloads · 30 days
65
5% of all-time downloads
gpt2-small-amharic-8k-128-v3
rasyosef/gpt2-small-amharic-8k-128-v3
gpt2-small-amharic-8k-128-v3 is a text generation model from rasyosef. Use it when you need the model to write or continue text. It is set up for transformers.
This is a smaller version of the gpt2 decoder transformer model pretrained from scratch for 1.5 days on 290 million tokens of Amharic text.
All-time downloads
1.3K
Public
Parameters
29.5M
1.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors118 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- Jul 1, 2024
- Updated
- Jul 4, 2024
- SHA
- 3839e961
Try a prompt
Model card
From the Hugging Face model README
gpt2-small-amharic-8k-128-v3
This is a smaller version of the gpt2 decoder transformer model pretrained from scratch for 1.5 days on 290 million tokens of Amharic text.
- It has 29.5 Million parameters
- The context size of this model is 128 tokens.
- It has the same tokenizer as gpt2, trained from scratch using the same dataset with a vocabulary size of 8192.
- This is a base model and hasn't undergone any supervised finetuing yet.
It achieves the following results on the evaluation set:
Loss: 3.59Perplexity: 36.23
Demo
You can use the following demo to generate text using gpt2-small-amharic. Please enter a prompt and click the Generate button to generate completions for the prompt.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Languages
- am
- Created
- Jul 1, 2024
- Updated
- Jul 4, 2024