Downloads · 30 days
106
3% of all-time downloads
NepaliGPT
Shushant/NepaliGPT
NepaliGPT is a text generation model from Shushant. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as bsd-3-clause-clear.
This is an experiment for developing a language generation model for the Nepali language. Causal Language Model which can predict the next possible tokens given a context in Nepali language.
All-time downloads
4.2K
Public
Parameters
88.2M
668 MB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin334 MB · 50%
Parameter types
How the weights are stored.
F3281.9M · 93%
Model card
From the Hugging Face model README
NepaliGPT: Nepali Language Generative Pretrained Transformer Model
This is an experiment for developing a language generation model for the Nepali language. Causal Language Model which can predict the next possible tokens given a context in Nepali language.
Dataset Used
A large corpus of 9.3 GB size has been collected from different sources on the internet. The sources include
- Nepali Books found online.
- Nepali News Article from Nepali news portals.
- Nepali text collected from different open source Nepali NLP datasets.
Hyperparameters Used
Learning rate -> 2e-5
Weight Decay -> 0.01
Number of training epochs -> 5 \
bf16 -> True
Base Model Architecture -> GPT-2 \
Training Results
It achieves the following results on the evaluation set:
| Training Loss | Validation Loss | Perplexity |
|---|---|---|
| 3.3968 | 3.2705 | 26.3245 |
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- bsd-3-clause-clear
- Languages
- ne
- Created
- Jul 13, 2023
- Updated
- Jul 13, 2023