fine-tuned-BioBARTv2-20-epochs-1024-input-256-output

tanatapanun/fine-tuned-BioBARTv2-20-epochs-1024-input-256-output

fine-tuned-BioBARTv2-20-epochs-1024-input-256-output is a machine learning model from tanatapanun. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

14% of all-time downloads

All-time downloads

Public

Parameters

166M

4.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors666 MB · 100%

At a glance

Library: transformers
Model type: bart
Access: Public
Created: Jan 29, 2024
Updated: Jan 29, 2024
SHA: c1529ffd

Base models

checkpoint_global_step_200000

Model card

From the Hugging Face model README

fine-tuned-BioBARTv2-20-epochs-1024-input-256-output

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.9612
Rouge1: 0.2101
Rouge2: 0.0415
Rougel: 0.1705
Rougelsum: 0.1695
Gen Len: 40.3

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	6.4708	0.1075	0.0008	0.079	0.0796	69.35
No log	2.0	302	1.1525	0.1053	0.0308	0.0902	0.0909	22.16
No log	3.0	453	0.9813	0.02	0.0	0.02	0.02	8.0
4.2222	4.0	604	0.9282	0.1047	0.0244	0.0791	0.0779	27.31
4.2222	5.0	755	0.8997	0.166	0.0316	0.1257	0.1248	73.57
4.2222	6.0	906	0.8912	0.1626	0.0288	0.1342	0.1327	37.82
0.7695	7.0	1057	0.8761	0.1756	0.0396	0.1377	0.1384	39.43
0.7695	8.0	1208	0.8787	0.155	0.0292	0.1225	0.1237	33.5
0.7695	9.0	1359	0.8834	0.1538	0.0268	0.1286	0.1276	27.75
0.5376	10.0	1510	0.8855	0.1924	0.0376	0.1542	0.1533	36.22
0.5376	11.0	1661	0.8915	0.1997	0.037	0.1607	0.1611	41.79
0.5376	12.0	1812	0.9022	0.1907	0.0289	0.1517	0.1533	43.94
0.5376	13.0	1963	0.9101	0.2054	0.0412	0.1603	0.1598	45.65
0.3808	14.0	2114	0.9172	0.1905	0.0405	0.1436	0.1441	44.98
0.3808	15.0	2265	0.9382	0.2121	0.0499	0.1729	0.1745	38.68
0.3808	16.0	2416	0.9437	0.1875	0.0387	0.1483	0.1482	37.93
0.2761	17.0	2567	0.9466	0.1935	0.0393	0.1513	0.1513	42.51
0.2761	18.0	2718	0.9518	0.2102	0.0478	0.1651	0.1664	40.2
0.2761	19.0	2869	0.9603	0.2074	0.0396	0.1691	0.1686	37.55
0.2257	20.0	3020	0.9612	0.2101	0.0415	0.1705	0.1695	40.3

Framework versions

Transformers 4.36.2
Pytorch 1.12.1+cu113
Datasets 2.16.1
Tokenizers 0.15.1

Library: transformers
Type: bart
Created: Jan 29, 2024
Updated: Jan 29, 2024