amazon_domain_pretrained_model

ltuzova/amazon_domain_pretrained_model

amazon_domain_pretrained_model is a fill-mask model from ltuzova. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

3% of all-time downloads

All-time downloads

290

Public

Repo size

2.5 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin499 MB · 100%

At a glance

Task: Fill-Mask
Library: transformers
License: mit
Model type: roberta
Access: Public
Created: Apr 6, 2024
Updated: Apr 10, 2024
SHA: b1ca9f16

Try a prompt

Model card

From the Hugging Face model README

amazon_domain_pretrained_model

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.4165

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 32
eval_batch_size: 64
seed: 42
gradient_accumulation_steps: 64
total_train_batch_size: 2048
optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.06
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0	10	1.6922
No log	0.01	20	1.6359
No log	0.01	30	1.6075
No log	0.02	40	1.5957
No log	0.02	50	1.5806
No log	0.03	60	1.5801
No log	0.03	70	1.5776
No log	0.04	80	1.5719
No log	0.04	90	1.5710
No log	0.05	100	1.5837
No log	0.05	110	1.5795
No log	0.06	120	1.5760
No log	0.06	130	1.5867
No log	0.07	140	1.5892
No log	0.07	150	1.5853
No log	0.08	160	1.5803
No log	0.08	170	1.5909
No log	0.09	180	1.5810
No log	0.09	190	1.5763
No log	0.1	200	1.5805
No log	0.1	210	1.5725
No log	0.1	220	1.5938
No log	0.11	230	1.5735
No log	0.11	240	1.5735
No log	0.12	250	1.5692
No log	0.12	260	1.5634
No log	0.13	270	1.5649
No log	0.13	280	1.5660
No log	0.14	290	1.5695
No log	0.14	300	1.5689
No log	0.15	310	1.5648
No log	0.15	320	1.5565
No log	0.16	330	1.5622
No log	0.16	340	1.5580
No log	0.17	350	1.5516
No log	0.17	360	1.5537
No log	0.18	370	1.5537
No log	0.18	380	1.5506
No log	0.19	390	1.5543
No log	0.19	400	1.5447
No log	0.19	410	1.5471
No log	0.2	420	1.5459
No log	0.2	430	1.5480
No log	0.21	440	1.5313
No log	0.21	450	1.5353
No log	0.22	460	1.5423
No log	0.22	470	1.5408
No log	0.23	480	1.5399
No log	0.23	490	1.5330
1.6837	0.24	500	1.5389
1.6837	0.24	510	1.5360
1.6837	0.25	520	1.5244
1.6837	0.25	530	1.5230
1.6837	0.26	540	1.5348
1.6837	0.26	550	1.5223
1.6837	0.27	560	1.5165
1.6837	0.27	570	1.5288
1.6837	0.28	580	1.5195
1.6837	0.28	590	1.5208
1.6837	0.29	600	1.5217
1.6837	0.29	610	1.5249
1.6837	0.29	620	1.5138
1.6837	0.3	630	1.5154
1.6837	0.3	640	1.5194
1.6837	0.31	650	1.5206
1.6837	0.31	660	1.5178
1.6837	0.32	670	1.5195
1.6837	0.32	680	1.5134
1.6837	0.33	690	1.5118
1.6837	0.33	700	1.5135
1.6837	0.34	710	1.5178
1.6837	0.34	720	1.5106
1.6837	0.35	730	1.5077
1.6837	0.35	740	1.5065
1.6837	0.36	750	1.5076
1.6837	0.36	760	1.5003
1.6837	0.37	770	1.5049
1.6837	0.37	780	1.5094
1.6837	0.38	790	1.5022
1.6837	0.38	800	1.4966
1.6837	0.38	810	1.5028
1.6837	0.39	820	1.4946
1.6837	0.39	830	1.5052
1.6837	0.4	840	1.4997
1.6837	0.4	850	1.5073
1.6837	0.41	860	1.4915
1.6837	0.41	870	1.5003
1.6837	0.42	880	1.4918
1.6837	0.42	890	1.4994
1.6837	0.43	900	1.4975
1.6837	0.43	910	1.4993
1.6837	0.44	920	1.4931
1.6837	0.44	930	1.4962
1.6837	0.45	940	1.4947
1.6837	0.45	950	1.4911
1.6837	0.46	960	1.4918
1.6837	0.46	970	1.4893
1.6837	0.47	980	1.4866
1.6837	0.47	990	1.4845
1.6086	0.48	1000	1.4829
1.6086	0.48	1010	1.4847
1.6086	0.48	1020	1.4801
1.6086	0.49	1030	1.4831
1.6086	0.49	1040	1.4860
1.6086	0.5	1050	1.4832
1.6086	0.5	1060	1.4814
1.6086	0.51	1070	1.4825
1.6086	0.51	1080	1.4780
1.6086	0.52	1090	1.4742
1.6086	0.52	1100	1.4821
1.6086	0.53	1110	1.4770
1.6086	0.53	1120	1.4730
1.6086	0.54	1130	1.4739
1.6086	0.54	1140	1.4718
1.6086	0.55	1150	1.4706
1.6086	0.55	1160	1.4729
1.6086	0.56	1170	1.4712
1.6086	0.56	1180	1.4699
1.6086	0.57	1190	1.4659
1.6086	0.57	1200	1.4685
1.6086	0.58	1210	1.4721
1.6086	0.58	1220	1.4716
1.6086	0.58	1230	1.4604
1.6086	0.59	1240	1.4619
1.6086	0.59	1250	1.4716
1.6086	0.6	1260	1.4643
1.6086	0.6	1270	1.4640
1.6086	0.61	1280	1.4616
1.6086	0.61	1290	1.4638
1.6086	0.62	1300	1.4605
1.6086	0.62	1310	1.4634
1.6086	0.63	1320	1.4580
1.6086	0.63	1330	1.4591
1.6086	0.64	1340	1.4597
1.6086	0.64	1350	1.4585
1.6086	0.65	1360	1.4555
1.6086	0.65	1370	1.4509
1.6086	0.66	1380	1.4591
1.6086	0.66	1390	1.4525
1.6086	0.67	1400	1.4479
1.6086	0.67	1410	1.4511
1.6086	0.67	1420	1.4545
1.6086	0.68	1430	1.4542
1.6086	0.68	1440	1.4469
1.6086	0.69	1450	1.4525
1.6086	0.69	1460	1.4452
1.6086	0.7	1470	1.4509
1.6086	0.7	1480	1.4524
1.6086	0.71	1490	1.4470
1.5617	0.71	1500	1.4479
1.5617	0.72	1510	1.4444
1.5617	0.72	1520	1.4485
1.5617	0.73	1530	1.4433
1.5617	0.73	1540	1.4380
1.5617	0.74	1550	1.4387
1.5617	0.74	1560	1.4383
1.5617	0.75	1570	1.4438
1.5617	0.75	1580	1.4338
1.5617	0.76	1590	1.4446
1.5617	0.76	1600	1.4376
1.5617	0.77	1610	1.4407
1.5617	0.77	1620	1.4384
1.5617	0.77	1630	1.4354
1.5617	0.78	1640	1.4344
1.5617	0.78	1650	1.4335
1.5617	0.79	1660	1.4364
1.5617	0.79	1670	1.4342
1.5617	0.8	1680	1.4319
1.5617	0.8	1690	1.4359
1.5617	0.81	1700	1.4389
1.5617	0.81	1710	1.4352
1.5617	0.82	1720	1.4324
1.5617	0.82	1730	1.4330
1.5617	0.83	1740	1.4281
1.5617	0.83	1750	1.4298
1.5617	0.84	1760	1.4243
1.5617	0.84	1770	1.4277
1.5617	0.85	1780	1.4253
1.5617	0.85	1790	1.4300
1.5617	0.86	1800	1.4272
1.5617	0.86	1810	1.4284
1.5617	0.86	1820	1.4293
1.5617	0.87	1830	1.4242
1.5617	0.87	1840	1.4267
1.5617	0.88	1850	1.4240
1.5617	0.88	1860	1.4193
1.5617	0.89	1870	1.4273
1.5617	0.89	1880	1.4174
1.5617	0.9	1890	1.4199
1.5617	0.9	1900	1.4239
1.5617	0.91	1910	1.4240
1.5617	0.91	1920	1.4201
1.5617	0.92	1930	1.4161
1.5617	0.92	1940	1.4222
1.5617	0.93	1950	1.4102
1.5617	0.93	1960	1.4177
1.5617	0.94	1970	1.4157
1.5617	0.94	1980	1.4100
1.5617	0.95	1990	1.4194
1.5215	0.95	2000	1.4232
1.5215	0.96	2010	1.4116
1.5215	0.96	2020	1.4243
1.5215	0.96	2030	1.4151
1.5215	0.97	2040	1.4202
1.5215	0.97	2050	1.4129
1.5215	0.98	2060	1.4138
1.5215	0.98	2070	1.4097
1.5215	0.99	2080	1.4143
1.5215	0.99	2090	1.4084
1.5215	1.0	2100	1.4132

Framework versions

Transformers 4.30.2
Pytorch 1.12.0+cu113
Datasets 2.13.2
Tokenizers 0.13.3

Task: Fill-Mask
Library: transformers
Type: roberta
License: mit
Created: Apr 6, 2024
Updated: Apr 10, 2024