BERTModified-rawbert-finetuned-wikitext-test

sreesha123456/BERTModified-rawbert-finetuned-wikitext-test

BERTModified-rawbert-finetuned-wikitext-test is a machine learning model from sreesha123456. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

Access

Public

Updated Mar 25, 2024

Parameters

138M

47.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors551 MB · 100%

At a glance

License: apache-2.0
Access: Public
Created: Mar 3, 2024
Updated: Mar 25, 2024
SHA: 9e39e9f0

Base models

Model card

From the Hugging Face model README

BERTModified-rawbert-finetuned-wikitext-test

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 21.1324
Precision: 0.0411
Recall: 0.0411
F1: 0.0411
Accuracy: 0.0411

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
21.2329	1.0	25	21.1260	0.0157	0.0157	0.0157	0.0157
18.2402	2.0	50	20.8705	0.0181	0.0181	0.0181	0.0181
15.7819	3.0	75	20.8017	0.0206	0.0206	0.0206	0.0206
13.5492	4.0	100	20.7470	0.0206	0.0206	0.0206	0.0206
11.836	5.0	125	20.7319	0.0254	0.0254	0.0254	0.0254
10.306	6.0	150	20.7540	0.0314	0.0314	0.0314	0.0314
9.0142	7.0	175	20.7665	0.0363	0.0363	0.0363	0.0363
7.991	8.0	200	20.8323	0.0363	0.0363	0.0363	0.0363
7.0936	9.0	225	20.9107	0.0387	0.0387	0.0387	0.0387
6.3742	10.0	250	20.9569	0.0399	0.0399	0.0399	0.0399
5.7236	11.0	275	20.9811	0.0375	0.0375	0.0375	0.0375
5.3262	12.0	300	21.0331	0.0435	0.0435	0.0435	0.0435
4.8222	13.0	325	21.0361	0.0387	0.0387	0.0387	0.0387
4.5049	14.0	350	21.0610	0.0363	0.0363	0.0363	0.0363
4.1877	15.0	375	21.0827	0.0387	0.0387	0.0387	0.0387
3.9705	16.0	400	21.1020	0.0435	0.0435	0.0435	0.0435
3.8091	17.0	425	21.0863	0.0435	0.0435	0.0435	0.0435
3.5978	18.0	450	21.1236	0.0447	0.0447	0.0447	0.0447
3.5991	19.0	475	21.1211	0.0435	0.0435	0.0435	0.0435
3.5433	20.0	500	21.1236	0.0435	0.0435	0.0435	0.0435

Framework versions

Transformers 4.39.1
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2

License: apache-2.0
Created: Mar 3, 2024
Updated: Mar 25, 2024