Downloads · 30 days
16
2% of all-time downloads
sinhala-gpt2
Ransaka/sinhala-gpt2
sinhala-gpt2 is a text generation model from Ransaka. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This particular model has undergone fine-tuning based on the gpt2 architecture, utilizing a dataset of Sinhala NEWS from various sources.
All-time downloads
667
Public
Repo size
64.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin510 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt2
- Access
- Public
- Created
- Mar 25, 2023
- Updated
- Sep 17, 2023
- SHA
- 80f1456c
Try a prompt
Model card
From the Hugging Face model README
sinhala-gpt2
This particular model has undergone fine-tuning based on the gpt2 architecture, utilizing a dataset of Sinhala NEWS from various sources.
Training procedure
The model was trained for 12+ hours on Kaggle GPUs.
Usage Details
from transformers import AutoTokenizer, AutoModelForCausalLM,pipeline
tokenizer = AutoTokenizer.from_pretrained("Ransaka/sinhala-gpt2")
model = AutoModelForCausalLM.from_pretrained("Ransaka/sinhala-gpt2")
generator("දුර")
or using git
git lfs install
git clone https://huggingface.co/Ransaka/sinhala-gpt2
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.0233 | 1.0 | 15323 | 2.3348 |
| 1.6938 | 2.0 | 30646 | 1.8377 |
| 1.4938 | 3.0 | 45969 | 1.6498 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- mit
- Languages
- si
- Created
- Mar 25, 2023
- Updated
- Sep 17, 2023