MBERT_revised_2e-06_64_0.1_0.01_110k

mbzuai-ugrip-statement-tuning/MBERT_revised_2e-06_64_0.1_0.01_110k

MBERT_revised_2e-06_64_0.1_0.01_110k is a text classification model from mbzuai-ugrip-statement-tuning. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

9% of all-time downloads

All-time downloads

Public

Parameters

178M

711 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors711 MB · 100%

At a glance

Task: Text Classification
Library: transformers
License: apache-2.0
Model type: bert
Access: Public
Created: Jun 12, 2024
Updated: Jun 12, 2024
SHA: 04a014ce

Try a prompt

Base models

google-bert/bert-base-multilingual-cased

Model card

From the Hugging Face model README

MBERT_revised-revised-outputs

This model is a fine-tuned version of google-bert/bert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4229
Accuracy: 0.7431
F1: 0.7564
Precision: 0.7276
Recall: 0.7876

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-06
train_batch_size: 64
eval_batch_size: 64
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.6935	0.7220	1000	0.6865	0.5423	0.6628	0.5286	0.8883
0.6351	1.4440	2000	0.5492	0.6788	0.7211	0.6435	0.8199
0.5285	2.1661	3000	0.4751	0.7106	0.7428	0.6753	0.8255
0.4727	2.8881	4000	0.4398	0.7220	0.7598	0.6753	0.8686
0.4405	3.6101	5000	0.4205	0.7293	0.7641	0.6838	0.8656
0.4221	4.3321	6000	0.4113	0.7339	0.7290	0.7525	0.7069
0.41	5.0542	7000	0.4145	0.7386	0.7700	0.6945	0.8640
0.3957	5.7762	8000	0.4018	0.7399	0.7807	0.6811	0.9145
0.388	6.4982	9000	0.3993	0.7416	0.7789	0.6872	0.8989
0.3807	7.2202	10000	0.3996	0.7413	0.7786	0.6870	0.8985
0.3734	7.9422	11000	0.3939	0.7459	0.7459	0.7553	0.7368
0.3719	8.6643	12000	0.3923	0.7455	0.7733	0.7043	0.8572
0.3625	9.3863	13000	0.3942	0.7405	0.7759	0.6894	0.8873
0.3646	10.1083	14000	0.4070	0.7422	0.7694	0.7032	0.8493
0.3578	10.8303	15000	0.3988	0.7435	0.7505	0.7396	0.7617
0.354	11.5523	16000	0.4031	0.7470	0.7616	0.7283	0.7980
0.3509	12.2744	17000	0.4089	0.7472	0.7476	0.7559	0.7394
0.3503	12.9964	18000	0.4029	0.7488	0.7676	0.7220	0.8195
0.3451	13.7184	19000	0.4127	0.7439	0.7449	0.7515	0.7384
0.3427	14.4404	20000	0.4196	0.7469	0.7684	0.7159	0.8291
0.3425	15.1625	21000	0.4168	0.7501	0.7736	0.7147	0.8429
0.3386	15.8845	22000	0.4131	0.7459	0.7507	0.7457	0.7559
0.3349	16.6065	23000	0.4279	0.7443	0.7472	0.7483	0.7460
0.336	17.3285	24000	0.4199	0.7441	0.7523	0.7378	0.7675
0.3341	18.0505	25000	0.4209	0.7449	0.7523	0.7400	0.7651
0.3333	18.7726	26000	0.4216	0.7461	0.7602	0.7284	0.7948
0.3323	19.4946	27000	0.4229	0.7431	0.7564	0.7276	0.7876

Framework versions

Transformers 4.41.2
Pytorch 2.3.1
Datasets 2.19.2
Tokenizers 0.19.1

Task: Text Classification
Library: transformers
Type: bert
License: apache-2.0
Created: Jun 12, 2024
Updated: Jun 12, 2024