NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

cj453/NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh is a machine learning model from cj453. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

12% of all-time downloads

All-time downloads

362

Public

Parameters

1.4B

5.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors5.7 GB · 100%

At a glance

License: other
Model type: opt
Access: Public
Created: Sep 18, 2024
Updated: Sep 18, 2024
SHA: 8ca8b6a7

Base models

facebook/opt-1.3b

Model card

From the Hugging Face model README

NumTrainEpochs10_SaveStrategiesno_reward_modeling_anthropic_hh

This model is a fine-tuned version of facebook/opt-1.3b on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.2160
Accuracy: 0.6289
Train Rewards/chosen: 13.3266
Train Rewards/rejected: -10.7412
Train Rewards/accuracies: 0.9925
Train Rewards/margins: 24.0678
Train Nll Loss: 1.9271
Train Logit Total Loss: 0.0395
Train Logit Loss: 0.0204
Rewards/chosen: 4.7138
Rewards/rejected: -1.7686
Rewards/accuracies: 0.6145
Rewards/margins: 6.4823
Nll Loss: 2.0087
Logit Total Loss: 3.2131
Logit Loss: 3.2252

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1.41e-05
train_batch_size: 4
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Nll Loss	Logit Total Loss	Logit Loss
0.7879	0.11	100	0.7507	0.4845	-0.1740	-0.1876	0.4714	0.0135	6.2659	0.7498	0.6941
0.7291	0.23	200	0.7331	0.5773	-0.2697	-0.3880	0.5645	0.1184	6.0096	0.7310	0.6777
0.6843	0.34	300	0.7057	0.5876	0.2058	-0.0389	0.5754	0.2448	3.9577	0.7039	0.6710
0.6773	0.46	400	0.6950	0.5918	-0.0097	-0.2138	0.5774	0.2041	4.2789	0.6968	0.6607
0.7071	0.57	500	0.7107	0.5918	0.7447	0.5198	0.5815	0.2249	4.4170	0.7087	0.6712
0.6881	0.69	600	0.6687	0.6186	-0.8010	-1.0541	0.6028	0.2531	3.2753	0.6671	0.6408
0.6871	0.8	700	0.6847	0.5753	-1.7064	-1.9330	0.5601	0.2266	3.7264	0.6839	0.6532
0.7125	0.91	800	0.6885	0.5814	-1.4574	-1.6521	0.5734	0.1947	4.3386	0.6851	0.6482
0.62	1.03	900	0.6955	0.6103	-1.4133	-1.8571	0.5964	0.4438	3.1332	0.6958	0.6712
0.5929	1.14	1000	0.6537	0.6371	-1.9413	-2.5254	0.6226	0.5841	2.9107	0.6524	0.6296
0.5825	1.26	1100	0.6749	0.6515	0.4669	-0.0787	0.6367	0.5455	2.9364	0.6732	0.6503
0.614	1.37	1200	0.6697	0.6351	0.1785	-0.2933	0.6258	0.4718	2.9997	0.6659	0.6423
0.5528	1.49	1300	0.6553	0.6268	-1.0780	-1.6306	0.6177	0.5526	2.9051	0.6504	0.6276
0.6501	1.6	1400	0.6379	0.6412	-1.6259	-2.1203	0.6306	0.4944	2.9085	0.6351	0.6121
0.545	1.71	1500	0.6640	0.6660	-0.3375	-1.1855	0.6560	0.8480	3.0934	0.6573	0.6327
0.6653	1.83	1600	0.6379	0.6639	-1.0663	-1.6961	0.6528	0.6298	2.8475	0.6376	0.6153
0.5792	1.94	1700	0.6447	0.6577	-0.0283	-0.6093	0.6480	0.5810	3.0457	0.6420	0.6178
0.2858	2.06	1800	0.9327	0.6330	1.7576	0.2131	0.6226	1.5445	2.8731	0.9216	0.9019
0.3404	2.17	1900	0.8438	0.6144	0.9326	-0.2443	0.6024	1.1769	2.7925	0.8418	0.8221
0.2734	2.29	2000	0.9082	0.6227	1.6779	0.4268	0.6125	1.2511	2.7991	0.8986	0.8794
0.2562	2.4	2100	0.9566	0.6371	2.2122	0.5184	0.6266	1.6937	2.7729	0.9522	0.9338
0.3796	2.51	2200	0.8839	0.6351	0.7900	-0.5311	0.6218	1.3211	2.7689	0.8720	0.8528
0.2316	2.63	2300	0.8741	0.6454	2.0133	0.5784	0.6359	1.4349	2.7465	0.8633	0.8443
0.3679	2.74	2400	0.8584	0.6515	-0.8628	-2.1801	0.6379	1.3173	2.7134	0.8483	0.8294
0.3384	2.86	2500	0.9165	0.6412	-0.9835	-2.3685	0.6294	1.3850	2.7084	0.9087	0.8905
0.3595	2.97	2600	0.9173	0.6454	0.3307	-1.0129	0.6347	1.3436	2.7089	0.9049	0.8867
0.1331	3.09	2700	1.4595	0.6557	0.6119	-2.1780	0.6468	2.7900	2.6967	1.4381	1.4254
0.1464	3.2	2800	1.4234	0.6351	5.4974	2.9945	0.6258	2.5029	2.6392	1.3999	1.3874
0.137	3.31	2900	1.4612	0.6474	3.1356	0.4400	0.6363	2.6956	2.6002	1.4435	1.4318
0.1593	3.43	3000	1.7826	0.6433	3.8280	0.7687	0.6282	3.0593	2.6206	1.7676	1.7590
0.0834	3.54	3100	1.5493	0.6474	2.4447	-0.2971	0.6355	2.7418	2.6296	1.5386	1.5275
0.136	3.66	3200	1.5847	0.6495	2.6691	-0.1416	0.6375	2.8108	2.6007	1.5701	1.5597
0.0859	3.77	3300	1.7114	0.6227	0.8690	-1.9033	0.6093	2.7723	2.5630	1.6942	1.6854
0.1705	3.89	3400	1.7792	0.6268	-1.4030	-4.0698	0.6121	2.6669	2.5917	1.7786	1.7704
0.1675	4.0	3500	2.1762	0.6268	-1.5886	-5.0180	0.6133	3.4294	2.5716	2.1579	2.1537
0.0589	4.11	3600	2.3409	0.6309	1.1330	-2.8993	0.6173	4.0323	2.4949	2.3055	2.3036
0.1014	4.23	3700	2.3221	0.6268	2.6255	-1.3486	0.6125	3.9741	2.4617	2.2985	2.2969
0.0697	4.34	3800	2.4256	0.6351	2.8885	-1.2680	0.6194	4.1565	2.4613	2.4010	2.4004
0.1687	4.46	3900	2.1905	0.6433	3.3404	-1.0572	0.6347	4.3976	2.4074	2.1582	2.1556
0.0315	4.57	4000	2.3170	0.6619	2.0050	-2.4036	0.6480	4.4086	2.4112	2.2812	2.2799
0.1071	4.69	4100	2.2205	0.6454	0.9399	-3.4755	0.6379	4.4154	2.3561	2.1998	2.1983
0.1342	4.8	4200	2.2640	0.6557	10.1640	5.7216	0.6419	4.4424	2.3536	2.2410	2.2399
0.0793	4.91	4300	2.0629	0.6495	-0.6830	-4.8288	0.6327	4.1458	2.3658	2.0407	2.0374
0.0587	5.03	4400	2.3862	0.6371	3.2076	-1.4161	0.6258	4.6238	2.3529	2.3625	2.3626
0.0433	5.14	4500	2.5409	0.6454	4.9940	0.1253	0.6286	4.8687	2.3166	2.5250	2.5271
0.0506	5.26	4600	2.5949	0.6557	6.7660	1.6624	0.6395	5.1035	2.2864	2.5983	2.6014
0.0506	5.37	4700	2.7389	0.6351	7.2608	2.0917	0.6226	5.1690	2.2691	2.7197	2.7243
0.0644	5.49	4800	2.8523	0.6309	2.3756	-2.9285	0.6173	5.3041	2.2594	2.8574	2.8634
0.0714	5.6	4900	2.5013	0.6309	2.5445	-2.3571	0.6206	4.9016	2.2422	2.5045	2.5072
0.1087	5.71	5000	2.6378	0.6227	-0.0320	-5.0243	0.6113	4.9923	2.2318	2.6303	2.6344
0.0874	5.83	5100	2.8088	0.6412	5.9816	0.6049	0.6278	5.3767	2.2257	2.7811	2.7867
0.0871	5.94	5200	2.4819	0.6433	7.2347	2.1895	0.6306	5.0451	2.2034	2.4679	2.4706
0.0331	6.06	5300	2.8775	0.6268	9.8380	4.4195	0.6145	5.4184	2.1978	2.8663	2.8730
0.024	6.17	5400	2.8923	0.6433	5.1441	-0.5990	0.6306	5.7431	2.1912	2.8713	2.8781
0.0354	6.29	5500	2.7626	0.6433	-1.4206	-6.9376	0.6315	5.5170	2.1826	2.7519	2.7577
0.0289	6.4	5600	2.8423	0.6371	7.1683	1.7904	0.6246	5.3779	2.1707	2.8182	2.8248
0.0389	6.51	5700	2.9096	0.6412	2.0666	-3.5386	0.6234	5.6052	2.1672	2.9140	2.9215
0.0245	6.63	5800	2.8677	0.6495	4.5194	-1.1798	0.6347	5.6992	2.1466	2.8461	2.8532
0.0804	6.74	5900	2.9668	0.6371	5.6766	-0.3308	0.6226	6.0074	2.1468	2.9437	2.9518
0.029	6.86	6000	3.0269	0.6371	3.9285	-2.2229	0.6226	6.1514	2.1305	2.9998	3.0086
0.0438	6.97	6100	2.8192	0.6639	2.2607	-4.3102	0.6476	6.5708	2.1277	2.8101	2.8170
0.0451	7.09	6200	2.8547	0.6577	2.5219	-3.4933	0.6395	6.0152	2.1111	2.8383	2.8456
0.0761	7.2	6300	2.9610	0.6536	4.7705	-1.5571	0.6435	6.3275	2.1023	2.9370	2.9454
0.0477	7.31	6400	2.8708	0.6619	2.7809	-3.7082	0.6488	6.4891	2.0958	2.8410	2.8485
0.0449	7.43	6500	3.0901	0.6619	5.8808	-0.8822	0.6496	6.7630	2.0873	3.0685	3.0784
0.0418	7.54	6600	2.9687	0.6371	2.2079	-4.1264	0.6206	6.3343	2.0853	2.9514	2.9602
0.0473	7.66	6700	2.9895	0.6351	2.4455	-3.8039	0.6206	6.2494	2.0790	2.9705	2.9795
0.0459	7.77	6800	3.0660	0.6392	4.6892	-1.6980	0.6254	6.3872	2.0757	3.0540	3.0638
0.045	7.89	6900	3.0811	0.6474	2.9687	-3.4595	0.6347	6.4282	2.0697	3.0561	3.0661
0.0493	8.0	7000	2.9549	0.6330	3.3733	-2.8947	0.6214	6.2680	2.0679	2.9435	2.9523
0.031	8.11	7100	2.9964	0.6330	4.2065	-2.1412	0.6206	6.3477	2.0650	2.9810	2.9903
0.0196	8.23	7200	3.0962	0.6371	4.8289	-1.6916	0.6246	6.5204	2.0550	3.0800	3.0904
0.0223	8.34	7300	3.0038	0.6392	2.7990	-3.5327	0.6246	6.3317	2.0497	2.9870	2.9965
0.0629	8.46	7400	3.0349	0.6351	5.2916	-0.8920	0.6206	6.1836	2.0453	3.0076	3.0173
0.0922	8.57	7500	3.0735	0.6227	1.5229	-4.6388	0.6105	6.1617	2.0409	3.0489	3.0591
0.0302	8.69	7600	3.1279	0.6289	1.4324	-4.7615	0.6185	6.1939	2.0355	3.1060	3.1168
0.0589	8.8	7700	3.1274	0.6412	4.6809	-1.6469	0.6306	6.3279	2.0298	3.1051	3.1159
0.0389	8.91	7800	3.0308	0.6330	4.8002	-1.3492	0.6206	6.1494	2.0277	3.0129	3.0229
0.0252	9.03	7900	3.0680	0.6330	5.0212	-1.1246	0.6165	6.1458	2.0236	3.0565	3.0670
0.0652	9.14	8000	3.1190	0.6351	4.3150	-1.9926	0.6165	6.3076	2.0196	3.1234	3.1345
0.0201	9.26	8100	3.1413	0.6289	4.7573	-1.5726	0.6165	6.3299	2.0164	3.1389	3.1503
0.0443	9.37	8200	3.1135	0.6247	4.3945	-1.9119	0.6125	6.3065	2.0140	3.1029	3.1139
0.0186	9.49	8300	3.1597	0.6289	3.7131	-2.6943	0.6165	6.4074	2.0114	3.1487	3.1602
0.0352	9.6	8400	3.1513	0.6247	3.9594	-2.4902	0.6085	6.4496	2.0100	3.1409	3.1523
0.0225	9.71	8500	3.1966	0.6227	4.9750	-1.5016	0.6125	6.4766	2.0095	3.1854	3.1973
0.0385	9.83	8600	3.2165	0.6268	4.9076	-1.6079	0.6125	6.5155	2.0094	3.2082	3.2203
0.0266	9.94	8700	3.2160	0.6289	4.7138	-1.7686	0.6145	6.4823	2.0087	3.2131	3.2252

Framework versions

Transformers 4.37.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.15.2

Type: opt
License: other
Created: Sep 18, 2024
Updated: Sep 18, 2024