fine-tuned-BioBARTv2-20-epochs-1024-input-320-output

tanatapanun/fine-tuned-BioBARTv2-20-epochs-1024-input-320-output

fine-tuned-BioBARTv2-20-epochs-1024-input-320-output is a machine learning model from tanatapanun. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

5% of all-time downloads

All-time downloads

Public

Parameters

166M

4.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors666 MB · 100%

At a glance

Library: transformers
Model type: bart
Access: Public
Created: Jan 30, 2024
Updated: Jan 30, 2024
SHA: f5938dad

Base models

checkpoint_global_step_200000

Model card

From the Hugging Face model README

fine-tuned-BioBARTv2-20-epochs-1024-input-320-output

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.7867
Rouge1: 0.1624
Rouge2: 0.0352
Rougel: 0.1185
Rougelsum: 0.1188
Gen Len: 37.64

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	6.3285	0.0632	0.0007	0.0552	0.0555	37.84
No log	2.0	302	0.9332	0.1075	0.0282	0.0825	0.0825	62.51
No log	3.0	453	0.8092	0.0826	0.0196	0.0629	0.0622	28.26
4.0641	4.0	604	0.7617	0.1106	0.0346	0.0814	0.0814	32.19
4.0641	5.0	755	0.7385	0.1359	0.0266	0.1043	0.1048	35.85
4.0641	6.0	906	0.7296	0.1507	0.0296	0.1099	0.1112	45.66
0.6482	7.0	1057	0.7225	0.1315	0.026	0.0978	0.0992	36.35
0.6482	8.0	1208	0.7165	0.1573	0.0302	0.1218	0.1222	42.68
0.6482	9.0	1359	0.7191	0.1445	0.0307	0.1155	0.1156	30.12
0.4567	10.0	1510	0.7281	0.1827	0.0423	0.1403	0.1408	47.87
0.4567	11.0	1661	0.7320	0.1603	0.0311	0.1193	0.1193	33.69
0.4567	12.0	1812	0.7395	0.1697	0.0357	0.1267	0.1267	46.9
0.4567	13.0	1963	0.7515	0.1442	0.0297	0.1064	0.1065	31.28
0.3275	14.0	2114	0.7549	0.1767	0.0306	0.1255	0.1259	49.23
0.3275	15.0	2265	0.7680	0.1475	0.0327	0.1054	0.1057	37.28
0.3275	16.0	2416	0.7760	0.1525	0.0337	0.1065	0.1072	38.87
0.2407	17.0	2567	0.7797	0.1543	0.039	0.1163	0.1168	38.67
0.2407	18.0	2718	0.7845	0.1794	0.0382	0.13	0.1305	38.47
0.2407	19.0	2869	0.7860	0.1645	0.0372	0.1218	0.1219	36.26
0.1982	20.0	3020	0.7867	0.1624	0.0352	0.1185	0.1188	37.64

Framework versions

Transformers 4.36.2
Pytorch 1.12.1+cu113
Datasets 2.16.1
Tokenizers 0.15.1

Library: transformers
Type: bart
Created: Jan 30, 2024
Updated: Jan 30, 2024