parity_lr5e-4_batch128_train1-24_eval25

mtzig/parity_lr5e-4_batch128_train1-24_eval25

parity_lr5e-4_batch128_train1-24_eval25 is a machine learning model from mtzig. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

21% of all-time downloads

All-time downloads

Public

Parameters

10.7M

85.3 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors42.6 MB · 100%

At a glance

Library: transformers
Model type: nanogpt
Access: Public
Created: Apr 8, 2025
Updated: Apr 8, 2025
SHA: 1432a2ca

Model card

From the Hugging Face model README

parity_lr5e-4_batch128_train1-24_eval25

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.6932
Accuracy: 0.503

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 128
eval_batch_size: 512
seed: 23452399
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0	0	3.1802	0.0
0.6348	0.0043	100	0.7304	0.4995
0.496	0.0085	200	0.7047	0.495
0.4587	0.0128	300	0.7099	0.503
0.5157	0.0171	400	0.6995	0.5008
0.4757	0.0213	500	0.7055	0.4977
0.4381	0.0256	600	0.7905	0.4994
0.4112	0.0299	700	0.7068	0.5029
0.4687	0.0341	800	0.6958	0.4993
0.6409	0.0384	900	0.6983	0.4973
0.4482	0.0427	1000	0.6928	0.4994
0.4199	0.0469	1100	0.6920	0.4829
0.6614	0.0512	1200	0.6997	0.4996
0.5764	0.0555	1300	0.6960	0.4987
0.6243	0.0597	1400	0.6934	0.5001
0.67	0.0640	1500	0.7071	0.4992
0.6491	0.0683	1600	0.6960	0.5008
0.6874	0.0725	1700	0.6911	0.4871
0.6815	0.0768	1800	0.7413	0.4992
0.6497	0.0811	1900	0.6955	0.4992
0.6686	0.0853	2000	0.6960	0.5008
0.6624	0.0896	2100	0.6995	0.4992
0.6589	0.0939	2200	0.7005	0.4992
0.6788	0.0981	2300	0.6958	0.4992
0.6665	0.1024	2400	0.6953	0.4992
0.6473	0.1067	2500	0.7014	0.4992
0.682	0.1109	2600	0.6971	0.4992
0.6642	0.1152	2700	0.7000	0.5008
0.6411	0.1195	2800	0.6955	0.4992
0.6611	0.1237	2900	0.6978	0.4992
0.6501	0.1280	3000	0.6939	0.4992
0.6535	0.1323	3100	0.6963	0.4992
0.6507	0.1365	3200	0.6933	0.4992
0.6675	0.1408	3300	0.6943	0.4992
0.6466	0.1451	3400	0.6935	0.5008
0.6515	0.1493	3500	0.7037	0.4992
0.6748	0.1536	3600	0.6942	0.4992
0.6409	0.1579	3700	0.6934	0.5008
0.6628	0.1621	3800	0.6968	0.4992
0.6386	0.1664	3900	0.6946	0.4992
0.6525	0.1707	4000	0.6954	0.4992
0.6857	0.1749	4100	0.6946	0.4992
0.65	0.1792	4200	0.6937	0.5008
0.6695	0.1835	4300	0.6942	0.5008
0.6676	0.1877	4400	0.6941	0.5008
0.6591	0.1920	4500	0.6972	0.5008
0.6537	0.1963	4600	0.6932	0.4992
0.6771	0.2005	4700	0.6935	0.5008
0.5977	0.2048	4800	0.6938	0.4992
0.6751	0.2091	4900	0.6938	0.5008
0.6611	0.2133	5000	0.6962	0.5008
0.6913	0.2176	5100	0.6932	0.4992
0.6444	0.2219	5200	0.6935	0.5008
0.6526	0.2261	5300	0.6939	0.4992
0.6554	0.2304	5400	0.6966	0.4992
0.6638	0.2347	5500	0.6956	0.5008
0.673	0.2389	5600	0.6933	0.5008
0.6423	0.2432	5700	0.6933	0.5008
0.672	0.2475	5800	0.6947	0.4992
0.6539	0.2517	5900	0.6956	0.4992
0.6319	0.2560	6000	0.6957	0.4992
0.6613	0.2603	6100	0.6934	0.497
0.6808	0.2645	6200	0.6996	0.5008
0.6866	0.2688	6300	0.6952	0.5008
0.6544	0.2731	6400	0.6936	0.4992
0.6663	0.2773	6500	0.6933	0.4992
0.6594	0.2816	6600	0.6938	0.4992
0.6618	0.2859	6700	0.6959	0.4992
0.6683	0.2901	6800	0.6939	0.4992
0.6371	0.2944	6900	0.6932	0.5008
0.6405	0.2987	7000	0.6947	0.4992
0.6831	0.3029	7100	0.6934	0.5008
0.6585	0.3072	7200	0.6934	0.4992
0.665	0.3115	7300	0.6942	0.5008
0.6593	0.3157	7400	0.6940	0.4992
0.6699	0.3200	7500	0.6956	0.5008
0.6724	0.3243	7600	0.6934	0.497
0.6669	0.3285	7700	0.6934	0.497
0.6518	0.3328	7800	0.6936	0.4992
0.676	0.3371	7900	0.6939	0.497
0.6865	0.3413	8000	0.6968	0.4992
0.676	0.3456	8100	0.6947	0.4992
0.6695	0.3499	8200	0.6933	0.5008
0.6756	0.3541	8300	0.6934	0.5008
0.6601	0.3584	8400	0.6933	0.497
0.627	0.3627	8500	0.6936	0.4992
0.6727	0.3669	8600	0.6936	0.497
0.6514	0.3712	8700	0.6939	0.5008
0.67	0.3755	8800	0.6943	0.4992
0.6805	0.3797	8900	0.6945	0.4992
0.6675	0.3840	9000	0.6937	0.497
0.6522	0.3883	9100	0.6937	0.497
0.6502	0.3925	9200	0.6935	0.5008
0.6392	0.3968	9300	0.6940	0.4992
0.6593	0.4011	9400	0.6935	0.497
0.6567	0.4053	9500	0.6937	0.4992
0.6888	0.4096	9600	0.6938	0.497
0.6795	0.4139	9700	0.6954	0.4992
0.6627	0.4181	9800	0.6940	0.497
0.6549	0.4224	9900	0.6936	0.497
0.6688	0.4267	10000	0.6959	0.4992
0.6685	0.4309	10100	0.6936	0.497
0.6833	0.4352	10200	0.6950	0.5008
0.6541	0.4395	10300	0.6943	0.497
0.6533	0.4437	10400	0.6945	0.497
0.6626	0.4480	10500	0.6948	0.497
0.6562	0.4523	10600	0.6941	0.497
0.6662	0.4565	10700	0.6939	0.497
0.6585	0.4608	10800	0.6943	0.497
0.6494	0.4651	10900	0.6935	0.497
0.6808	0.4693	11000	0.6935	0.497
0.6788	0.4736	11100	0.6945	0.4992
0.6333	0.4779	11200	0.6940	0.4992
0.6637	0.4821	11300	0.6950	0.4992
0.6522	0.4864	11400	0.6931	0.4994
0.653	0.4907	11500	0.6942	0.4992
0.6459	0.4949	11600	0.6935	0.497
0.6547	0.4992	11700	0.6933	0.497
0.6374	0.5035	11800	0.6940	0.4992
0.6451	0.5077	11900	0.6934	0.4992
0.686	0.5120	12000	0.6934	0.497
0.6487	0.5163	12100	0.6933	0.497
0.645	0.5205	12200	0.6945	0.4992
0.6364	0.5248	12300	0.6932	0.4992
0.6806	0.5291	12400	0.6955	0.4992
0.6406	0.5333	12500	0.6938	0.4992
0.6705	0.5376	12600	0.6942	0.5008
0.6512	0.5419	12700	0.6960	0.4992
0.6752	0.5461	12800	0.6932	0.5008
0.6762	0.5504	12900	0.6947	0.5008
0.6423	0.5547	13000	0.6933	0.497
0.6543	0.5589	13100	0.6934	0.5008
0.6535	0.5632	13200	0.6933	0.497
0.6601	0.5675	13300	0.6932	0.497
0.6724	0.5717	13400	0.6932	0.4992
0.6531	0.5760	13500	0.6935	0.5008
0.6449	0.5803	13600	0.6951	0.5008
0.656	0.5845	13700	0.6933	0.4992
0.612	0.5888	13800	0.6962	0.5008
0.6618	0.5931	13900	0.6932	0.4992
0.6624	0.5973	14000	0.6934	0.5008
0.6862	0.6016	14100	0.6946	0.4992
0.669	0.6059	14200	0.6933	0.497
0.6514	0.6101	14300	0.6936	0.5008
0.6685	0.6144	14400	0.6934	0.5008
0.6426	0.6187	14500	0.6934	0.5008
0.6354	0.6229	14600	0.6934	0.4992
0.6744	0.6272	14700	0.6934	0.5008
0.6609	0.6315	14800	0.6931	0.503
0.6784	0.6357	14900	0.6932	0.5008
0.6813	0.6400	15000	0.6938	0.5008
0.6871	0.6443	15100	0.6932	0.4992
0.651	0.6485	15200	0.6933	0.5008
0.6518	0.6528	15300	0.6932	0.5008
0.6601	0.6571	15400	0.6936	0.4992
0.6222	0.6613	15500	0.6932	0.4992
0.689	0.6656	15600	0.6933	0.4992
0.6485	0.6699	15700	0.6936	0.5008
0.6439	0.6741	15800	0.6933	0.5008
0.6786	0.6784	15900	0.6931	0.503
0.6377	0.6827	16000	0.6934	0.5008
0.6447	0.6869	16100	0.6932	0.5008
0.654	0.6912	16200	0.6934	0.5008
0.6317	0.6955	16300	0.6933	0.5008
0.6414	0.6997	16400	0.6932	0.5008
0.6556	0.7040	16500	0.6934	0.5008
0.65	0.7083	16600	0.6934	0.497
0.6511	0.7125	16700	0.6932	0.497
0.6405	0.7168	16800	0.6932	0.5008
0.6476	0.7211	16900	0.6933	0.5008
0.6543	0.7253	17000	0.6932	0.503
0.6758	0.7296	17100	0.6934	0.5008
0.6489	0.7339	17200	0.6934	0.4992
0.6396	0.7381	17300	0.6931	0.5024
0.6496	0.7424	17400	0.6931	0.503
0.6559	0.7467	17500	0.6931	0.503
0.6517	0.7509	17600	0.6936	0.5008
0.6662	0.7552	17700	0.6931	0.503
0.6735	0.7595	17800	0.6936	0.497
0.6632	0.7637	17900	0.6937	0.497
0.623	0.7680	18000	0.6941	0.497
0.6651	0.7723	18100	0.6934	0.497
0.6469	0.7765	18200	0.6934	0.4992
0.6542	0.7808	18300	0.6958	0.5008
0.6319	0.7850	18400	0.6933	0.4994
0.6524	0.7893	18500	0.6934	0.4992
0.602	0.7936	18600	0.6932	0.5008
0.5984	0.7978	18700	0.6934	0.4992
0.6227	0.8021	18800	0.6932	0.4992
0.6199	0.8064	18900	0.6932	0.5008
0.6047	0.8106	19000	0.6935	0.5008
0.6261	0.8149	19100	0.6934	0.5008
0.5993	0.8192	19200	0.6939	0.5008
0.5848	0.8234	19300	0.6932	0.4992
0.6157	0.8277	19400	0.6932	0.497
0.6149	0.8320	19500	0.6932	0.4972
0.6331	0.8362	19600	0.6932	0.5008
0.672	0.8405	19700	0.6936	0.5008
0.6172	0.8448	19800	0.6936	0.5008
0.6183	0.8490	19900	0.6937	0.5008
0.5877	0.8533	20000	0.6932	0.5008
0.5834	0.8576	20100	0.6933	0.5007
0.6132	0.8618	20200	0.6932	0.5008
0.6063	0.8661	20300	0.6932	0.5024
0.6058	0.8704	20400	0.6934	0.5005
0.6285	0.8746	20500	0.6933	0.4991
0.5617	0.8789	20600	0.6948	0.5008
0.5896	0.8832	20700	0.6933	0.503
0.579	0.8874	20800	0.6932	0.503
0.5868	0.8917	20900	0.6932	0.503
0.5423	0.8960	21000	0.6933	0.4937
0.5743	0.9002	21100	0.6937	0.4992
0.5245	0.9045	21200	0.6934	0.5008
0.5347	0.9088	21300	0.6933	0.4948
0.6894	0.9130	21400	0.6933	0.4965
0.5917	0.9173	21500	0.6938	0.5008
0.5395	0.9216	21600	0.6932	0.4999
0.5591	0.9258	21700	0.6933	0.4992
0.5542	0.9301	21800	0.6938	0.5008
0.5796	0.9344	21900	0.6936	0.4968
0.6201	0.9386	22000	0.6935	0.4962
0.5537	0.9429	22100	0.6935	0.4942
0.722	0.9472	22200	0.6931	0.503
0.5787	0.9514	22300	0.6938	0.5008
0.5721	0.9557	22400	0.6937	0.5007
0.5812	0.9600	22500	0.6933	0.4928
0.5312	0.9642	22600	0.6935	0.5008
0.6068	0.9685	22700	0.6934	0.4975
0.5745	0.9728	22800	0.6932	0.4958
0.5653	0.9770	22900	0.6932	0.4986
0.5967	0.9813	23000	0.6932	0.5029
0.5685	0.9856	23100	0.6932	0.503
0.5604	0.9898	23200	0.6932	0.503
0.5551	0.9941	23300	0.6932	0.503
0.5532	0.9984	23400	0.6932	0.503

Framework versions

Transformers 4.46.0
Pytorch 2.5.1
Datasets 3.1.0
Tokenizers 0.20.1

Library: transformers
Type: nanogpt
Created: Apr 8, 2025
Updated: Apr 8, 2025