GerMedBert_ATTR_V02_BRONCO

BachelorThesis/GerMedBert_ATTR_V02_BRONCO

GerMedBert_ATTR_V02_BRONCO is a text classification model from BachelorThesis. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.

This model is a fine-tuned version of GerMedBERT/medbert-512 on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.0593 - F1 Score: 0.8187 - Precision: 0.8235 - Recall: 0.8140 - Accur…

Downloads · 30 days

2% of all-time downloads

All-time downloads

313

Public

Parameters

109M

436 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors436 MB · 100%

At a glance

Task: Text Classification
Library: transformers
License: apache-2.0
Model type: bert
Access: Public
Created: Jun 2, 2024
Updated: Jun 2, 2024
SHA: 895a1207

Base models

GerMedBERT/medbert-512

Datasets

bigbio/bronco

Model card

From the Hugging Face model README

GerMedBert_ATTR_V02_BRONCO

This model is a fine-tuned version of GerMedBERT/medbert-512 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.0593
F1 Score: 0.8187
Precision: 0.8235
Recall: 0.8140
Accuracy: 0.8993
Num Input Tokens Seen: 4204246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Input Tokens Seen
No log	0.25	81	0.1949	0.0	1.0	0.0	0.7240	175856
0.206	0.5	162	0.1196	0.4123	0.8393	0.2733	0.7847	349280
0.206	0.75	243	0.0945	0.6296	0.8673	0.4942	0.8351	526096
0.1019	1.0	324	0.0792	0.7584	0.8	0.7209	0.8698	702407
0.1019	1.25	405	0.0697	0.7874	0.7784	0.7965	0.8785	877047
0.0715	1.5	486	0.0697	0.7778	0.7447	0.8140	0.8715	1052647
0.0715	1.75	567	0.0673	0.7568	0.7826	0.7326	0.8715	1227287
0.0638	2.0	648	0.0680	0.7781	0.7358	0.8256	0.8715	1400330
0.0638	2.25	729	0.0622	0.7965	0.8084	0.7849	0.8941	1577786
0.0445	2.5	810	0.0593	0.8012	0.7943	0.8081	0.8906	1751722
0.0445	2.75	891	0.0583	0.8023	0.8023	0.8023	0.8906	1927258
0.0396	3.0	972	0.0579	0.8161	0.8068	0.8256	0.8993	2100749
0.0396	3.25	1053	0.0598	0.8125	0.7944	0.8314	0.8941	2276989
0.0289	3.5	1134	0.0592	0.8036	0.8232	0.7849	0.8941	2451501
0.0289	3.75	1215	0.0585	0.7954	0.7886	0.8023	0.8906	2628573
0.0271	4.0	1296	0.0571	0.8171	0.8034	0.8314	0.8993	2802576
0.0271	4.25	1377	0.0581	0.8235	0.8333	0.8140	0.9045	2978368
0.0194	4.5	1458	0.0619	0.7978	0.7717	0.8256	0.8837	3154544
0.0194	4.75	1539	0.0612	0.8048	0.8323	0.7791	0.8958	3330400
0.0193	5.0	1620	0.0585	0.8059	0.8155	0.7965	0.8958	3505555
0.0193	5.25	1701	0.0587	0.8187	0.8235	0.8140	0.9010	3680771
0.0153	5.5	1782	0.0592	0.8242	0.8171	0.8314	0.9010	3856947
0.0153	5.75	1863	0.0592	0.8163	0.8187	0.8140	0.8993	4030371
0.0146	6.0	1944	0.0593	0.8187	0.8235	0.8140	0.8993	4204246

Framework versions

Transformers 4.40.1
Pytorch 2.3.0+cu121
Datasets 2.19.0
Tokenizers 0.19.1

Task: Text Classification
Library: transformers
Type: bert
License: apache-2.0
Languages: de
Created: Jun 2, 2024
Updated: Jun 2, 2024