add_lr5e-4_batch128_train17_evaltrain

mtzig/add_lr5e-4_batch128_train17_evaltrain

add_lr5e-4_batch128_train17_evaltrain is a machine learning model from mtzig. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

20% of all-time downloads

All-time downloads

Public

Parameters

10.7M

42.6 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors42.6 MB · 100%

At a glance

Library: transformers
Model type: nanogpt
Access: Public
Created: Apr 9, 2025
Updated: Apr 9, 2025
SHA: 640985d1

Model card

From the Hugging Face model README

add_lr5e-4_batch128_train17_evaltrain

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.7110
Accuracy: 0.3371

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 128
eval_batch_size: 512
seed: 23452399
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0	0	2.7171	0.0
2.3534	0.0032	50	2.3677	0.0
2.3382	0.0064	100	2.5148	0.0
2.311	0.0096	150	2.3971	0.0
2.3392	0.0128	200	2.5468	0.0
2.3034	0.016	250	2.3131	0.0
2.2137	0.0192	300	2.4709	0.0
2.1959	0.0224	350	2.3693	0.0
2.2031	0.0256	400	2.5114	0.0
2.2015	0.0288	450	2.5978	0.0
2.182	0.032	500	2.5588	0.0
2.2006	0.0352	550	2.4233	0.0
2.1606	0.0384	600	2.5245	0.0
2.1524	0.0416	650	2.4452	0.0
2.1044	0.0448	700	2.3655	0.0
2.0603	0.048	750	2.3375	0.0
2.183	0.0512	800	2.3313	0.0
2.0273	0.0544	850	2.4725	0.0
2.033	0.0576	900	2.4836	0.0
1.9766	0.0608	950	2.3963	0.0
1.8946	0.064	1000	2.3595	0.0
1.8097	0.0672	1050	2.1780	0.0
1.6859	0.0704	1100	2.6667	0.0002
1.8248	0.0736	1150	2.7052	0.0
1.5832	0.0768	1200	2.3183	0.0001
1.6588	0.08	1250	2.5069	0.0
1.4987	0.0832	1300	2.3781	0.0
1.4534	0.0864	1350	2.5671	0.0
1.6611	0.0896	1400	2.7870	0.0
1.6325	0.0928	1450	2.3185	0.0001
1.5752	0.096	1500	2.5077	0.0
1.3086	0.0992	1550	2.9031	0.0
1.621	0.1024	1600	3.0532	0.0
1.3097	0.1056	1650	3.2942	0.0
1.5028	0.1088	1700	2.3666	0.0
1.3305	0.112	1750	3.2486	0.0
1.4764	0.1152	1800	2.9466	0.0
1.4808	0.1184	1850	2.8717	0.0
1.3628	0.1216	1900	3.1647	0.0
1.3876	0.1248	1950	2.8130	0.0004
1.2997	0.128	2000	2.7729	0.0001
1.3024	0.1312	2050	2.7103	0.0003
1.3018	0.1344	2100	3.1862	0.0002
1.4855	0.1376	2150	2.7146	0.0002
1.4855	0.1408	2200	3.5807	0.0
1.4745	0.144	2250	2.3634	0.0002
1.3088	0.1472	2300	3.3057	0.0
1.2341	0.1504	2350	2.7851	0.0006
1.3885	0.1536	2400	2.7682	0.0001
1.366	0.1568	2450	3.0570	0.0
1.3261	0.16	2500	3.0537	0.0003
1.2401	0.1632	2550	3.4798	0.0002
1.2072	0.1664	2600	3.2757	0.0002
1.2796	0.1696	2650	2.8221	0.0002
1.2952	0.1728	2700	3.2149	0.0
1.2149	0.176	2750	3.5343	0.0
1.2852	0.1792	2800	3.9293	0.0
1.1779	0.1824	2850	3.8437	0.0002
1.2693	0.1856	2900	3.3776	0.0003
1.12	0.1888	2950	3.4571	0.0004
1.227	0.192	3000	3.1741	0.0007
1.1679	0.1952	3050	3.2316	0.0005
1.2782	0.1984	3100	3.0204	0.0002
1.1612	0.2016	3150	3.7882	0.0003
1.3976	0.2048	3200	2.7002	0.0004
1.2371	0.208	3250	3.1928	0.0009
1.2195	0.2112	3300	3.4656	0.0006
1.2678	0.2144	3350	2.7147	0.0008
1.1537	0.2176	3400	3.4452	0.0005
1.3463	0.2208	3450	3.5082	0.0003
1.1471	0.224	3500	3.4308	0.0007
1.2255	0.2272	3550	3.2201	0.0008
1.1674	0.2304	3600	3.5609	0.0001
1.2189	0.2336	3650	3.8242	0.0005
1.3446	0.2368	3700	4.4741	0.0
1.176	0.24	3750	3.4270	0.0005
1.1728	0.2432	3800	3.8401	0.0004
1.2267	0.2464	3850	3.6995	0.0004
1.1773	0.2496	3900	4.4688	0.0004
1.1547	0.2528	3950	3.9891	0.0003
1.2737	0.256	4000	4.3630	0.0
1.1536	0.2592	4050	4.0964	0.0005
1.2369	0.2624	4100	4.3463	0.0001
1.2146	0.2656	4150	3.9598	0.0
1.1365	0.2688	4200	3.1020	0.0004
1.2101	0.272	4250	3.7791	0.0004
1.2298	0.2752	4300	3.8624	0.0004
1.1941	0.2784	4350	3.9779	0.0008
1.2113	0.2816	4400	3.5294	0.001
1.1585	0.2848	4450	4.2826	0.0
1.2869	0.288	4500	3.8736	0.0
1.0868	0.2912	4550	4.3815	0.001
1.3051	0.2944	4600	4.4681	0.0026
1.1707	0.2976	4650	4.8402	0.0018
1.1597	0.3008	4700	5.0568	0.0013
1.3069	0.304	4750	4.1207	0.0001
1.1646	0.3072	4800	4.1790	0.0034
1.3297	0.3104	4850	4.0990	0.0024
1.0493	0.3136	4900	4.5623	0.0061
1.1153	0.3168	4950	4.6378	0.0004
0.8784	0.32	5000	4.1497	0.0086
0.896	0.3232	5050	4.6908	0.0051
0.822	0.3264	5100	4.1919	0.0108
0.6781	0.3296	5150	4.9788	0.0049
0.6793	0.3328	5200	5.3065	0.0017
0.2857	0.336	5250	4.4342	0.0095
0.5048	0.3392	5300	4.5093	0.0141
0.5522	0.3424	5350	4.9845	0.0066
0.1214	0.3456	5400	5.0081	0.0199
0.414	0.3488	5450	4.4191	0.0256
0.2786	0.352	5500	4.3840	0.0052
0.3123	0.3552	5550	5.4591	0.002
0.274	0.3584	5600	5.1043	0.0332
0.1928	0.3616	5650	3.9612	0.0748
0.2678	0.3648	5700	3.7207	0.0439
0.5844	0.368	5750	4.0531	0.0151
0.0855	0.3712	5800	4.5378	0.034
0.7164	0.3744	5850	3.1294	0.016
0.3368	0.3776	5900	4.4172	0.0133
0.2381	0.3808	5950	3.8615	0.0243
0.4482	0.384	6000	3.5239	0.0453
0.2669	0.3872	6050	4.2445	0.0398
0.0958	0.3904	6100	4.3887	0.0105
0.1462	0.3936	6150	3.7110	0.0554
0.0327	0.3968	6200	3.3010	0.0623
0.0222	0.4	6250	3.9386	0.0939
0.0559	0.4032	6300	3.9364	0.0755
0.1217	0.4064	6350	4.3939	0.0215
0.1358	0.4096	6400	3.1975	0.0703
0.0646	0.4128	6450	2.9846	0.0645
0.0236	0.416	6500	3.3172	0.0596
0.1173	0.4192	6550	2.8748	0.1073
0.0582	0.4224	6600	2.9948	0.1046
0.0573	0.4256	6650	3.3605	0.1123
0.0326	0.4288	6700	3.2537	0.1046
0.0731	0.432	6750	3.1120	0.1109
0.0399	0.4352	6800	3.4842	0.0556
0.0109	0.4384	6850	2.5963	0.1168
0.2186	0.4416	6900	2.8680	0.0756
0.0877	0.4448	6950	2.5039	0.1407
0.0176	0.448	7000	2.2619	0.1053
0.0349	0.4512	7050	3.0717	0.1474
0.0375	0.4544	7100	3.3271	0.0761
0.1062	0.4576	7150	2.8235	0.163
0.0576	0.4608	7200	2.6568	0.056
0.0061	0.464	7250	3.3304	0.1348
0.085	0.4672	7300	2.6189	0.2183
0.0104	0.4704	7350	3.0011	0.1506
0.0045	0.4736	7400	2.6107	0.1811
0.052	0.4768	7450	2.6883	0.2167
0.124	0.48	7500	2.4655	0.0957
0.0016	0.4832	7550	2.3609	0.194
0.0096	0.4864	7600	3.1718	0.1336
0.0028	0.4896	7650	2.8083	0.1593
0.009	0.4928	7700	2.3825	0.1434
0.0348	0.496	7750	2.7852	0.1706
0.0177	0.4992	7800	2.6883	0.1856
0.0234	0.5024	7850	2.6150	0.201
0.0089	0.5056	7900	2.2804	0.1624
0.0194	0.5088	7950	2.8231	0.1422
0.0085	0.512	8000	3.3137	0.1384
0.0095	0.5152	8050	2.5829	0.1891
0.0065	0.5184	8100	2.7052	0.2005
0.2807	0.5216	8150	2.8978	0.1373
0.0423	0.5248	8200	2.6956	0.1769
0.0041	0.528	8250	2.6078	0.2284
0.0193	0.5312	8300	2.9127	0.2202
0.0036	0.5344	8350	3.1227	0.1996
0.0182	0.5376	8400	2.8784	0.1508
0.0321	0.5408	8450	3.2844	0.1415
0.0112	0.544	8500	2.7068	0.2207
0.0053	0.5472	8550	2.4179	0.249
0.0019	0.5504	8600	2.6199	0.213
0.0137	0.5536	8650	2.2050	0.1843
0.0103	0.5568	8700	2.7119	0.1982
0.0029	0.56	8750	2.5797	0.2548
0.0055	0.5632	8800	2.8202	0.1532
0.0011	0.5664	8850	2.6418	0.1814
0.0105	0.5696	8900	2.1258	0.2824
0.0052	0.5728	8950	2.6633	0.1998
0.0017	0.576	9000	2.9766	0.2135
0.0009	0.5792	9050	2.6429	0.2699
0.0141	0.5824	9100	2.2798	0.2971
0.0047	0.5856	9150	2.6866	0.2594
0.0001	0.5888	9200	2.7058	0.2383
0.0009	0.592	9250	2.2642	0.3119
0.0009	0.5952	9300	2.5964	0.2719
0.001	0.5984	9350	2.6379	0.2989
0.0001	0.6016	9400	2.1677	0.3591
0.0	0.6048	9450	2.3099	0.3358
0.0002	0.608	9500	2.3010	0.3246
0.0001	0.6112	9550	3.0135	0.2213
0.0008	0.6144	9600	2.8845	0.2961
0.0023	0.6176	9650	2.5845	0.2819
0.0014	0.6208	9700	2.4775	0.2596
0.0013	0.624	9750	2.3568	0.2443
0.0028	0.6272	9800	3.1175	0.2227
0.0002	0.6304	9850	3.7130	0.1188
0.0037	0.6336	9900	2.6004	0.3015
0.0054	0.6368	9950	2.2830	0.285
0.0004	0.64	10000	2.7812	0.2507
0.0001	0.6432	10050	2.8292	0.2596
0.004	0.6464	10100	2.5658	0.2914
0.0004	0.6496	10150	3.1035	0.1841
0.0007	0.6528	10200	2.1120	0.3746
0.0	0.656	10250	2.1145	0.3639
0.0	0.6592	10300	2.3072	0.3395
0.0	0.6624	10350	2.3622	0.338
0.0	0.6656	10400	2.3874	0.3396
0.0	0.6688	10450	2.4089	0.3384
0.0	0.672	10500	2.4243	0.3382
0.0	0.6752	10550	2.4334	0.3391
0.0	0.6784	10600	2.4398	0.34
0.0	0.6816	10650	2.4449	0.3419
0.0	0.6848	10700	2.4922	0.3307
0.0	0.688	10750	2.4956	0.3321
0.0	0.6912	10800	2.4984	0.3334
0.0	0.6944	10850	2.4563	0.3403
0.0	0.6976	10900	2.4635	0.3406
0.0	0.7008	10950	2.4701	0.3411
0.0	0.704	11000	2.4758	0.3416
0.0	0.7072	11050	2.4812	0.3415
0.0	0.7104	11100	2.4867	0.342
0.0	0.7136	11150	2.4943	0.3418
0.0	0.7168	11200	2.4993	0.342
0.0	0.72	11250	2.5051	0.3418
0.0	0.7232	11300	2.5138	0.3412
0.0	0.7264	11350	2.5205	0.3407
0.0	0.7296	11400	2.5264	0.3405
0.0	0.7328	11450	2.5294	0.3408
0.0	0.736	11500	2.5340	0.3414
0.0	0.7392	11550	2.5386	0.3412
0.0	0.7424	11600	2.5438	0.3408
0.0	0.7456	11650	2.5572	0.3395
0.0	0.7488	11700	2.5618	0.3395
0.0	0.752	11750	2.5651	0.3395
0.0	0.7552	11800	2.5781	0.3389
0.0	0.7584	11850	2.5829	0.3393
0.0	0.7616	11900	2.5863	0.3391
0.0	0.7648	11950	2.5888	0.339
0.0	0.768	12000	2.5926	0.3392
0.0	0.7712	12050	2.5956	0.3392
0.0	0.7744	12100	2.5989	0.3392
0.0	0.7776	12150	2.5874	0.3418
0.0	0.7808	12200	2.5917	0.3423
0.0	0.784	12250	2.5990	0.3419
0.0	0.7872	12300	2.6083	0.3414
0.0	0.7904	12350	2.6119	0.3414
0.0	0.7936	12400	2.6158	0.341
0.0	0.7968	12450	2.6200	0.3406
0.0	0.8	12500	2.6229	0.3405
0.0	0.8032	12550	2.6263	0.3404
0.0	0.8064	12600	2.6291	0.3404
0.0	0.8096	12650	2.6321	0.3404
0.0	0.8128	12700	2.6357	0.3404
0.0	0.816	12750	2.6385	0.3402
0.0	0.8192	12800	2.6414	0.3402
0.0	0.8224	12850	2.6432	0.34
0.0	0.8256	12900	2.6456	0.3399
0.0	0.8288	12950	2.6501	0.3395
0.0	0.832	13000	2.6534	0.3393
0.0	0.8352	13050	2.6565	0.3386
0.0	0.8384	13100	2.6587	0.3386
0.0	0.8416	13150	2.6603	0.3387
0.0	0.8448	13200	2.6623	0.3388
0.0	0.848	13250	2.6647	0.3385
0.0	0.8512	13300	2.6670	0.3384
0.0	0.8544	13350	2.6690	0.3384
0.0	0.8576	13400	2.6709	0.3382
0.0	0.8608	13450	2.6727	0.3382
0.0	0.864	13500	2.6755	0.338
0.0	0.8672	13550	2.6775	0.338
0.0	0.8704	13600	2.6790	0.3379
0.0	0.8736	13650	2.6808	0.338
0.0	0.8768	13700	2.6824	0.3378
0.0	0.88	13750	2.6857	0.3374
0.0	0.8832	13800	2.6883	0.3372
0.0	0.8864	13850	2.6901	0.3372
0.0	0.8896	13900	2.6918	0.3371
0.0	0.8928	13950	2.6928	0.3372
0.0	0.896	14000	2.6944	0.337
0.0	0.8992	14050	2.6957	0.337
0.0	0.9024	14100	2.6968	0.337
0.0	0.9056	14150	2.6978	0.337
0.0	0.9088	14200	2.6990	0.3371
0.0	0.912	14250	2.7000	0.3371
0.0	0.9152	14300	2.7011	0.337
0.0	0.9184	14350	2.7019	0.337
0.0	0.9216	14400	2.7029	0.337
0.0	0.9248	14450	2.7037	0.337
0.0	0.928	14500	2.7046	0.3369
0.0	0.9312	14550	2.7055	0.3369
0.0	0.9344	14600	2.7061	0.3369
0.0	0.9376	14650	2.7068	0.3369
0.0	0.9408	14700	2.7074	0.3369
0.0	0.944	14750	2.7079	0.337
0.0	0.9472	14800	2.7084	0.337
0.0	0.9504	14850	2.7088	0.337
0.0	0.9536	14900	2.7092	0.337
0.0	0.9568	14950	2.7095	0.3371
0.0	0.96	15000	2.7099	0.3371
0.0	0.9632	15050	2.7101	0.3371
0.0	0.9664	15100	2.7103	0.3371
0.0	0.9696	15150	2.7105	0.3371
0.0	0.9728	15200	2.7107	0.3371
0.0	0.976	15250	2.7108	0.3371
0.0	0.9792	15300	2.7109	0.3371
0.0	0.9824	15350	2.7109	0.3371
0.0	0.9856	15400	2.7110	0.3371
0.0	0.9888	15450	2.7110	0.3371
0.0	0.992	15500	2.7110	0.3371
0.0	0.9952	15550	2.7110	0.3371
0.0	0.9984	15600	2.7110	0.3371

Framework versions

Transformers 4.46.0
Pytorch 2.5.1
Datasets 3.1.0
Tokenizers 0.20.1

Library: transformers
Type: nanogpt
Created: Apr 9, 2025
Updated: Apr 9, 2025