Downloads · 30 days
4
15% of all-time downloads
fine-tuned-BioBART-20-epochs-test
tanatapanun/fine-tuned-BioBART-20-epochs-test
fine-tuned-BioBART-20-epochs-test is a machine learning model from tanatapanun. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
should probably proofread and complete it, then remove this comment. --
All-time downloads
27
Public
Parameters
169M
21.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors675 MB · 100%
At a glance
- Library
- transformers
- Model type
- bart
- Access
- Public
- Created
- Jan 22, 2024
- Updated
- Jan 22, 2024
- SHA
- c1f1253c
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
fine-tuned-BioBART-20-epochs-test
This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.1997
- Rouge1: 0.0956
- Rouge2: 0.0145
- Rougel: 0.0591
- Rougelsum: 0.0593
- Gen Len: 217.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 0.7878 | 1.0 | 1201 | 0.2002 | 0.0888 | 0.0237 | 0.0689 | 0.0691 | 224.36 |
| 0.2064 | 2.0 | 2402 | 0.1817 | 0.02 | 0.0 | 0.02 | 0.02 | 8.0 |
| 0.1708 | 3.0 | 3603 | 0.1638 | 0.03 | 0.0 | 0.03 | 0.03 | 5.0 |
| 0.136 | 4.0 | 4804 | 0.1576 | 0.0228 | 0.0036 | 0.0232 | 0.023 | 10.0 |
| 0.1346 | 5.0 | 6005 | 0.1559 | 0.0631 | 0.018 | 0.0592 | 0.0591 | 11.0 |
| 0.097 | 6.0 | 7206 | 0.1573 | 0.0928 | 0.0177 | 0.0784 | 0.079 | 20.0 |
| 0.086 | 7.0 | 8407 | 0.1607 | 0.0638 | 0.0086 | 0.0522 | 0.0523 | 21.0 |
| 0.0638 | 8.0 | 9608 | 0.1649 | 0.0228 | 0.0036 | 0.0232 | 0.023 | 10.0 |
| 0.0425 | 9.0 | 10809 | 0.1690 | 0.064 | 0.0198 | 0.0578 | 0.0579 | 20.0 |
| 0.0359 | 10.0 | 12010 | 0.1726 | 0.1024 | 0.0157 | 0.0817 | 0.0817 | 49.0 |
| 0.0262 | 11.0 | 13211 | 0.1771 | 0.0868 | 0.0198 | 0.0787 | 0.0792 | 20.0 |
| 0.0204 | 12.0 | 14412 | 0.1819 | 0.0977 | 0.0104 | 0.0748 | 0.075 | 43.0 |
| 0.0156 | 13.0 | 15613 | 0.1852 | 0.066 | 0.0094 | 0.0509 | 0.051 | 43.0 |
| 0.0131 | 14.0 | 16814 | 0.1885 | 0.1068 | 0.018 | 0.0726 | 0.0725 | 135.0 |
| 0.0105 | 15.0 | 18015 | 0.1915 | 0.0967 | 0.0248 | 0.0784 | 0.0787 | 30.0 |
| 0.009 | 16.0 | 19216 | 0.1950 | 0.104 | 0.0221 | 0.0791 | 0.079 | 73.0 |
| 0.0081 | 17.0 | 20417 | 0.1962 | 0.0967 | 0.0248 | 0.0784 | 0.0787 | 30.0 |
| 0.0077 | 18.0 | 21618 | 0.1978 | 0.0903 | 0.0084 | 0.0567 | 0.0567 | 174.0 |
| 0.0068 | 19.0 | 22819 | 0.1991 | 0.0896 | 0.0107 | 0.055 | 0.055 | 174.0 |
| 0.0064 | 20.0 | 24020 | 0.1997 | 0.0956 | 0.0145 | 0.0591 | 0.0593 | 217.0 |
Framework versions
- Transformers 4.36.2
- Pytorch 1.12.1+cu113
- Datasets 2.16.1
- Tokenizers 0.15.0
- Library
- transformers
- Type
- bart
- Created
- Jan 22, 2024
- Updated
- Jan 22, 2024