bert-base-uncased-finetuned-github_cybersecurity_READMEs

sickcell/bert-base-uncased-finetuned-github_cybersecurity_READMEs

bert-base-uncased-finetuned-github_cybersecurity_READMEs is a fill-mask model from sickcell. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

45% of all-time downloads

All-time downloads

Public

Parameters

110M

4.4 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors438 MB · 100%

At a glance

Task: Fill-Mask
Library: transformers
License: apache-2.0
Model type: bert
Access: Public
Created: Apr 12, 2024
Updated: Apr 14, 2024
SHA: e9bcb032

Try a prompt

Base models

Model card

From the Hugging Face model README

bert-base-uncased-finetuned-github_cybersecurity_READMEs

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.2291
Accuracy: 0.6479

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 1000
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.97	14	4.1856	0.4305
No log	2.0	29	4.3178	0.4090
No log	2.97	43	4.0734	0.4342
No log	4.0	58	4.0470	0.4332
No log	4.97	72	4.0668	0.4270
No log	6.0	87	3.9068	0.4390
No log	6.97	101	3.8466	0.4468
No log	8.0	116	3.8330	0.4535
No log	8.97	130	3.7238	0.4516
No log	10.0	145	3.8113	0.4446
No log	10.97	159	3.6681	0.4607
No log	12.0	174	3.5627	0.4679
No log	12.97	188	3.4540	0.4794
No log	14.0	203	3.5997	0.4707
No log	14.97	217	3.4362	0.4860
No log	16.0	232	3.5471	0.4740
No log	16.97	246	3.4968	0.4803
No log	18.0	261	3.2938	0.4985
No log	18.97	275	3.4207	0.4765
No log	20.0	290	3.3869	0.4970
No log	20.97	304	3.3062	0.5012
No log	22.0	319	3.3184	0.4917
No log	22.97	333	3.2132	0.5136
No log	24.0	348	3.2027	0.5074
No log	24.97	362	3.3251	0.4923
No log	26.0	377	3.1569	0.5108
No log	26.97	391	3.0947	0.5194
No log	28.0	406	3.0470	0.5206
No log	28.97	420	3.0662	0.5182
No log	30.0	435	3.0845	0.5191
No log	30.97	449	3.0681	0.5219
No log	32.0	464	2.9902	0.5263
No log	32.97	478	2.8970	0.5448
No log	34.0	493	2.9269	0.5341
3.629	34.97	507	2.8605	0.5519
3.629	36.0	522	2.8657	0.5431
3.629	36.97	536	2.9391	0.5407
3.629	38.0	551	2.8960	0.5437
3.629	38.97	565	2.8819	0.5466
3.629	40.0	580	2.7555	0.5633
3.629	40.97	594	2.7425	0.5555
3.629	42.0	609	2.7960	0.5615
3.629	42.97	623	2.7382	0.5630
3.629	44.0	638	2.7967	0.5580
3.629	44.97	652	2.6611	0.5781
3.629	46.0	667	2.6877	0.5722
3.629	46.97	681	2.7917	0.5609
3.629	48.0	696	2.7029	0.5696
3.629	48.97	710	2.7408	0.5618
3.629	50.0	725	2.6450	0.5772
3.629	50.97	739	2.5569	0.5883
3.629	52.0	754	2.6646	0.5795
3.629	52.97	768	2.6803	0.5729
3.629	54.0	783	2.6233	0.5847
3.629	54.97	797	2.6027	0.5842
3.629	56.0	812	2.4090	0.6034
3.629	56.97	826	2.4978	0.6011
3.629	58.0	841	2.5106	0.5944
3.629	58.97	855	2.5039	0.5912
3.629	60.0	870	2.5792	0.5824
3.629	60.97	884	2.4764	0.6065
3.629	62.0	899	2.5348	0.6036
3.629	62.97	913	2.5338	0.6022
3.629	64.0	928	2.4646	0.6130
3.629	64.97	942	2.4532	0.6066
3.629	66.0	957	2.4526	0.6073
3.629	66.97	971	2.5369	0.5992
3.629	68.0	986	2.4170	0.6181
2.5556	68.97	1000	2.4493	0.6078
2.5556	70.0	1015	2.3939	0.6159
2.5556	70.97	1029	2.4793	0.6049
2.5556	72.0	1044	2.3225	0.6286
2.5556	72.97	1058	2.3551	0.6212
2.5556	74.0	1073	2.4702	0.6075
2.5556	74.97	1087	2.3489	0.6311
2.5556	76.0	1102	2.3455	0.6198
2.5556	76.97	1116	2.4500	0.6145
2.5556	78.0	1131	2.3223	0.6332
2.5556	78.97	1145	2.4375	0.6065
2.5556	80.0	1160	2.2743	0.6291
2.5556	80.97	1174	2.3255	0.6295
2.5556	82.0	1189	2.3785	0.6237
2.5556	82.97	1203	2.2722	0.6344
2.5556	84.0	1218	2.2392	0.6407
2.5556	84.97	1232	2.2322	0.6361
2.5556	86.0	1247	2.2206	0.6496
2.5556	86.97	1261	2.2419	0.6345
2.5556	88.0	1276	2.1919	0.6492
2.5556	88.97	1290	2.2616	0.6433
2.5556	90.0	1305	2.2227	0.6417
2.5556	90.97	1319	2.2847	0.6447
2.5556	92.0	1334	2.2916	0.6339
2.5556	92.97	1348	2.2684	0.6410
2.5556	94.0	1363	2.2432	0.6440
2.5556	94.97	1377	2.2510	0.6462
2.5556	96.0	1392	2.2970	0.6363
2.5556	96.55	1400	2.2197	0.6423

Framework versions

Transformers 4.40.0.dev0
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2

Task: Fill-Mask
Library: transformers
Type: bert
License: apache-2.0
Created: Apr 12, 2024
Updated: Apr 14, 2024