model_mask30p_100epoch

tupolev4/model_mask30p_100epoch

model_mask30p_100epoch is a fill-mask model from tupolev4. Use it when you need the model to fill a missing word. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

26% of all-time downloads

All-time downloads

Public

Parameters

43.5M

14.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors174 MB · 100%

At a glance

Task: Fill-Mask
Library: transformers
Model type: distilbert
Access: Public
Created: Dec 10, 2023
Updated: Dec 10, 2023
SHA: 0a0524c9

Try a prompt

Model card

From the Hugging Face model README

model_mask30p_100epoch

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.2550

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	57	2.6725
No log	2.0	114	2.6084
No log	3.0	171	2.6114
No log	4.0	228	2.6452
No log	5.0	285	2.6414
No log	6.0	342	2.5859
No log	7.0	399	2.6358
No log	8.0	456	2.6219
2.6069	9.0	513	2.6005
2.6069	10.0	570	2.5896
2.6069	11.0	627	2.5994
2.6069	12.0	684	2.5500
2.6069	13.0	741	2.5967
2.6069	14.0	798	2.5714
2.6069	15.0	855	2.5651
2.6069	16.0	912	2.5307
2.6069	17.0	969	2.3990
2.5079	18.0	1026	2.3268
2.5079	19.0	1083	2.3078
2.5079	20.0	1140	2.3268
2.5079	21.0	1197	2.3278
2.5079	22.0	1254	2.3542
2.5079	23.0	1311	2.3773
2.5079	24.0	1368	2.3548
2.5079	25.0	1425	2.3574
2.5079	26.0	1482	2.3350
2.2916	27.0	1539	2.2998
2.2916	28.0	1596	2.3144
2.2916	29.0	1653	2.3098
2.2916	30.0	1710	2.3253
2.2916	31.0	1767	2.3219
2.2916	32.0	1824	2.2956
2.2916	33.0	1881	2.3017
2.2916	34.0	1938	2.3071
2.2916	35.0	1995	2.3131
2.2288	36.0	2052	2.3041
2.2288	37.0	2109	2.2773
2.2288	38.0	2166	2.2502
2.2288	39.0	2223	2.2330
2.2288	40.0	2280	2.2147
2.2288	41.0	2337	2.1744
2.2288	42.0	2394	2.1358
2.2288	43.0	2451	2.1509
2.1326	44.0	2508	2.1167
2.1326	45.0	2565	2.0464
2.1326	46.0	2622	2.0284
2.1326	47.0	2679	2.0077
2.1326	48.0	2736	1.9298
2.1326	49.0	2793	1.8966
2.1326	50.0	2850	1.8917
2.1326	51.0	2907	1.7883
2.1326	52.0	2964	1.6490
1.8676	53.0	3021	1.6729
1.8676	54.0	3078	1.5494
1.8676	55.0	3135	1.5023
1.8676	56.0	3192	1.3941
1.8676	57.0	3249	1.3139
1.8676	58.0	3306	1.2355
1.8676	59.0	3363	1.1424
1.8676	60.0	3420	1.0589
1.8676	61.0	3477	0.9061
1.3493	62.0	3534	0.8806
1.3493	63.0	3591	0.7774
1.3493	64.0	3648	0.6681
1.3493	65.0	3705	0.6103
1.3493	66.0	3762	0.5802
1.3493	67.0	3819	0.4806
1.3493	68.0	3876	0.4749
1.3493	69.0	3933	0.4502
1.3493	70.0	3990	0.4365
0.7176	71.0	4047	0.3917
0.7176	72.0	4104	0.3864
0.7176	73.0	4161	0.3690
0.7176	74.0	4218	0.3657
0.7176	75.0	4275	0.3494
0.7176	76.0	4332	0.3160
0.7176	77.0	4389	0.3182
0.7176	78.0	4446	0.2680
0.4147	79.0	4503	0.3013
0.4147	80.0	4560	0.2942
0.4147	81.0	4617	0.2767
0.4147	82.0	4674	0.3180
0.4147	83.0	4731	0.2372
0.4147	84.0	4788	0.2922
0.4147	85.0	4845	0.2649
0.4147	86.0	4902	0.2603
0.4147	87.0	4959	0.2553
0.3218	88.0	5016	0.2187
0.3218	89.0	5073	0.2584
0.3218	90.0	5130	0.2299
0.3218	91.0	5187	0.2611
0.3218	92.0	5244	0.2511
0.3218	93.0	5301	0.2506
0.3218	94.0	5358	0.2380
0.3218	95.0	5415	0.2522
0.3218	96.0	5472	0.2313
0.2876	97.0	5529	0.2121
0.2876	98.0	5586	0.2049
0.2876	99.0	5643	0.1957
0.2876	100.0	5700	0.2581

Framework versions

Transformers 4.35.2
Pytorch 2.1.0+cu118
Datasets 2.15.0
Tokenizers 0.15.0

Task: Fill-Mask
Library: transformers
Type: distilbert
Created: Dec 10, 2023
Updated: Dec 10, 2023