Downloads · 30 days
12
3% of all-time downloads
gpt_16_5_5.6e-5
nlp04/gpt_16_5_5.6e-5
gpt_16_5_5.6e-5 is a text generation model from nlp04. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
478
Public
Repo size
1.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin513 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- gpt2
- Access
- Public
- Created
- Jan 27, 2023
- Updated
- Jan 27, 2023
- SHA
- ff47999b
Try a prompt
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
gpt_16_5_5.6e-5
This model is a fine-tuned version of skt/kogpt2-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 4.0127
- Rouge1: 0.6626
- Rouge2: 0.1168
- Rougel: 0.6456
- Bleu1: 0.0
- Bleu2: 0.0
- Bleu3: 0.0
- Bleu4: 0.0
- Gen Len: 1.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Bleu1 | Bleu2 | Bleu3 | Bleu4 | Gen Len |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 3.1501 | 0.38 | 1000 | 4.1034 | 0.6908 | 0.1016 | 0.6736 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 3.1688 | 0.76 | 2000 | 4.0936 | 0.6447 | 0.1117 | 0.6372 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.8769 | 1.13 | 3000 | 4.0668 | 0.7186 | 0.1365 | 0.7012 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.8633 | 1.51 | 4000 | 4.0363 | 0.625 | 0.1187 | 0.6116 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.8928 | 1.89 | 5000 | 4.0112 | 0.6599 | 0.1237 | 0.6476 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.6186 | 2.27 | 6000 | 4.0204 | 0.6953 | 0.1259 | 0.6761 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.6653 | 2.64 | 7000 | 4.0044 | 0.6795 | 0.1163 | 0.6627 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.5418 | 3.02 | 8000 | 4.0183 | 0.602 | 0.1154 | 0.5883 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.4621 | 3.4 | 9000 | 4.0149 | 0.5889 | 0.101 | 0.578 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.4651 | 3.78 | 10000 | 3.9979 | 0.604 | 0.1098 | 0.5943 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.3334 | 4.15 | 11000 | 4.0226 | 0.6352 | 0.0993 | 0.6211 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.2859 | 4.53 | 12000 | 4.0152 | 0.6782 | 0.1125 | 0.6593 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
| 2.314 | 4.91 | 13000 | 4.0127 | 0.6626 | 0.1168 | 0.6456 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.9.0+cu102
- Datasets 2.8.0
- Tokenizers 0.13.2
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- License
- cc-by-nc-sa-4.0
- Created
- Jan 27, 2023
- Updated
- Jan 27, 2023