BioELECTRA-LitCovid-v1.3h

sofia-todeschini/BioELECTRA-LitCovid-v1.3h

BioELECTRA-LitCovid-v1.3h is a text classification model from sofia-todeschini. Use it when you need a label for a piece of text. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

6% of all-time downloads

All-time downloads

124

Public

Repo size

14.9 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin438 MB · 100%

At a glance

Task: Text Classification
Library: transformers
Model type: electra
Access: Public
Created: Mar 11, 2024
Updated: Mar 12, 2024
SHA: 3f06a5c9

Try a prompt

Model card

From the Hugging Face model README

BioELECTRA-LitCovid-v1.3h

This model is a fine-tuned version of kamalkraj/bioelectra-base-discriminator-pubmed on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.7785
Hamming loss: 0.0198
F1 micro: 0.8361
F1 macro: 0.3559
F1 weighted: 0.8787
F1 samples: 0.8727
Precision micro: 0.7527
Precision macro: 0.2860
Precision weighted: 0.8332
Precision samples: 0.8513
Recall micro: 0.9403
Recall macro: 0.7383
Recall weighted: 0.9403
Recall samples: 0.9483
Roc Auc: 0.9614
Accuracy: 0.6748

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1866747178469669
num_epochs: 5
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Hamming loss	F1 micro	F1 macro	F1 weighted	F1 samples	Precision micro	Precision macro	Precision weighted	Precision samples	Recall micro	Recall macro	Recall weighted	Recall samples	Roc Auc	Accuracy
1.5951	1.0	2272	0.6305	0.0577	0.6142	0.2231	0.7544	0.7325	0.4787	0.1794	0.7016	0.6957	0.8568	0.7122	0.8568	0.8824	0.9020	0.3878
1.1968	2.0	4544	0.4865	0.0353	0.7393	0.2825	0.8470	0.8156	0.6122	0.2305	0.7890	0.7790	0.9330	0.7538	0.9330	0.9449	0.9497	0.5560
0.9573	3.0	6816	0.5637	0.0247	0.8033	0.3292	0.8474	0.8430	0.7019	0.2574	0.7851	0.8066	0.9389	0.7380	0.9389	0.9470	0.9582	0.5918
0.7604	4.0	9088	0.6811	0.0206	0.8306	0.3558	0.8726	0.8675	0.7441	0.2835	0.8239	0.8438	0.9400	0.7574	0.9400	0.9483	0.9608	0.6561
0.4404	5.0	11360	0.7785	0.0198	0.8361	0.3559	0.8787	0.8727	0.7527	0.2860	0.8332	0.8513	0.9403	0.7383	0.9403	0.9483	0.9614	0.6748

Framework versions

Transformers 4.28.0
Pytorch 2.1.0+cu121
Datasets 2.18.0
Tokenizers 0.13.3

Task: Text Classification
Library: transformers
Type: electra
Created: Mar 11, 2024
Updated: Mar 12, 2024