Downloads · 30 days
71
5% of all-time downloads
gpt2-tigrinya-medium
luel/gpt2-tigrinya-medium
gpt2-tigrinya-medium is a text generation model from luel. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is a GPT-2 model trained from scratch on Tigrinya text data. It was trained on 20.6 million tokens, primarily from news sources.
All-time downloads
1.3K
Public
Parameters
51.9M
623 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin208 MB · 49%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Nov 2, 2024
- Updated
- Nov 8, 2024
- SHA
- 3bd863e0
Model card
From the Hugging Face model README
Model Card for GPT-2 Tigrinya Medium
Model Summary
This is a GPT-2 model trained from scratch on Tigrinya text data. It was trained on 20.6 million tokens, primarily from news sources.
Model Description
- Model type: GPT-2
- Language: Tigrinya (ትግርኛ)
- Finetuned from model: Trained from scratch (no pre-training)
Model Architecture
- Parameters: 51.9M
- Context Window: 128 tokens
- Vocabulary Size: 52,000
Training Details
- Training regime: fp16 mixed precision
- Number of Epochs: 12
- Batch Size: 6 (with gradient accumulation steps of 8)
- Learning Rate: 5e-4
Evaluation
- Training Perplexity: 28.6
- Training Loss: 3.12
Usage
from transformers import pipeline
# Load the model
generator = pipeline('text-generation', model='luel/gpt2-tigrinya-medium')
prompt = "ክልል ትግራይ"
# Generate text
text = generator(prompt, max_length=100)[0]['generated_text']
print(text)
Limitations
- Limited context window of 128 tokens.
- Best suited for medium-length Tigrinya text generation.
- Outputs should be reviewed for accuracy.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- ti
- Created
- Nov 2, 2024
- Updated
- Nov 8, 2024