fine-tuned-BioBART-50-epochs-1024-input-128-output

tanatapanun/fine-tuned-BioBART-50-epochs-1024-input-128-output

fine-tuned-BioBART-50-epochs-1024-input-128-output is a machine learning model from tanatapanun. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

10% of all-time downloads

All-time downloads

Public

Parameters

139M

8.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors558 MB · 100%

At a glance

Library: transformers
License: apache-2.0
Model type: bart
Access: Public
Created: Jan 25, 2024
Updated: Jan 25, 2024
SHA: 24c88392

Base models

GanjinZero/biobart-base

Model card

From the Hugging Face model README

fine-tuned-BioBART-50-epochs-1024-input-128-output

This model is a fine-tuned version of GanjinZero/biobart-base on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.9109
Rouge1: 0.1191
Rouge2: 0.0252
Rougel: 0.105
Rougelsum: 0.1059
Gen Len: 16.2

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	151	8.7986	0.0	0.0	0.0	0.0	14.54
No log	2.0	302	4.6009	0.007	0.0022	0.0066	0.0067	4.73
No log	3.0	453	1.9851	0.1025	0.0246	0.091	0.0906	13.95
6.1578	4.0	604	1.7001	0.0763	0.0172	0.0666	0.0674	10.25
6.1578	5.0	755	1.6023	0.1303	0.0277	0.1167	0.1164	15.08
6.1578	6.0	906	1.5322	0.0795	0.0176	0.0732	0.0736	14.54
1.4113	7.0	1057	1.4998	0.0972	0.0241	0.0839	0.0838	13.47
1.4113	8.0	1208	1.4808	0.0992	0.0238	0.0894	0.0898	14.28
1.4113	9.0	1359	1.4964	0.1249	0.0214	0.111	0.1106	12.36
0.8834	10.0	1510	1.4858	0.1459	0.0363	0.1235	0.1237	15.63
0.8834	11.0	1661	1.4990	0.1578	0.0403	0.1379	0.139	15.92
0.8834	12.0	1812	1.5210	0.1327	0.0253	0.1212	0.1209	15.11
0.8834	13.0	1963	1.5381	0.1372	0.038	0.1255	0.1251	15.45
0.5229	14.0	2114	1.5559	0.1383	0.0348	0.1263	0.1263	16.49
0.5229	15.0	2265	1.5824	0.1509	0.0369	0.1336	0.1325	15.78
0.5229	16.0	2416	1.6369	0.128	0.0298	0.1176	0.1185	14.12
0.2708	17.0	2567	1.6393	0.1362	0.0429	0.1229	0.1229	15.77
0.2708	18.0	2718	1.6599	0.1521	0.0402	0.1329	0.1333	15.34
0.2708	19.0	2869	1.6705	0.1293	0.0265	0.1165	0.1166	16.51
0.1203	20.0	3020	1.6943	0.141	0.0289	0.1273	0.1275	15.69
0.1203	21.0	3171	1.6969	0.1253	0.0337	0.1081	0.1085	16.35
0.1203	22.0	3322	1.7431	0.1319	0.0272	0.1185	0.1185	15.63
0.1203	23.0	3473	1.7434	0.1357	0.0343	0.1253	0.125	16.39
0.0509	24.0	3624	1.7507	0.1375	0.0325	0.1233	0.1231	16.79
0.0509	25.0	3775	1.7776	0.1222	0.0328	0.1121	0.1121	16.18
0.0509	26.0	3926	1.7733	0.1265	0.0216	0.1166	0.117	16.25
0.0257	27.0	4077	1.8001	0.1238	0.0239	0.1116	0.1113	16.44
0.0257	28.0	4228	1.7955	0.1173	0.0221	0.103	0.1046	16.64
0.0257	29.0	4379	1.8143	0.1311	0.0273	0.1186	0.1183	16.78
0.0164	30.0	4530	1.8108	0.1331	0.0296	0.1219	0.1226	15.64
0.0164	31.0	4681	1.8184	0.1245	0.0339	0.1134	0.1143	16.55
0.0164	32.0	4832	1.8545	0.1101	0.0217	0.0982	0.0998	16.09
0.0164	33.0	4983	1.8550	0.1421	0.0322	0.1292	0.1296	16.07
0.0117	34.0	5134	1.8573	0.1309	0.0292	0.1192	0.1193	16.0
0.0117	35.0	5285	1.8453	0.1254	0.0238	0.1133	0.1139	16.55
0.0117	36.0	5436	1.8724	0.1167	0.0241	0.1024	0.1035	15.89
0.0089	37.0	5587	1.8761	0.1345	0.0275	0.1206	0.1208	15.87
0.0089	38.0	5738	1.8772	0.1338	0.0301	0.1216	0.1228	16.78
0.0089	39.0	5889	1.8654	0.134	0.0264	0.1193	0.1196	16.85
0.0071	40.0	6040	1.8812	0.129	0.0287	0.1181	0.1177	16.12
0.0071	41.0	6191	1.8838	0.1238	0.0274	0.1134	0.1134	16.29
0.0071	42.0	6342	1.8752	0.1334	0.0262	0.1209	0.1214	16.66
0.0071	43.0	6493	1.8993	0.1238	0.0254	0.1111	0.1113	16.31
0.0056	44.0	6644	1.8963	0.1279	0.0346	0.1133	0.1154	16.07
0.0056	45.0	6795	1.9079	0.1225	0.0261	0.108	0.1084	16.09
0.0056	46.0	6946	1.9132	0.129	0.025	0.1157	0.1154	16.26
0.0045	47.0	7097	1.9120	0.1419	0.0362	0.1275	0.1278	15.78
0.0045	48.0	7248	1.9069	0.1316	0.0253	0.1161	0.1165	16.38
0.0045	49.0	7399	1.9099	0.1206	0.0259	0.1074	0.1077	16.32
0.0041	50.0	7550	1.9109	0.1191	0.0252	0.105	0.1059	16.2

Framework versions

Transformers 4.36.2
Pytorch 1.12.1+cu113
Datasets 2.16.1
Tokenizers 0.15.0

Library: transformers
Type: bart
License: apache-2.0
Created: Jan 25, 2024
Updated: Jan 25, 2024