phi-2-dpo-full-hydrox-safe

yihang7/phi-2-dpo-full-hydrox-safe

phi-2-dpo-full-hydrox-safe is a text generation model from yihang7. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

5% of all-time downloads

All-time downloads

335

Public

Parameters

2.8B

5.6 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors5.6 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: mit
Model type: phi
Access: Public
Created: Jan 25, 2024
Updated: Jan 26, 2024
SHA: d9912c7b

Try a prompt

Base models

microsoft/phi-2

Model card

From the Hugging Face model README

phi-2-dpo-full-hydrox-safe

This model is a fine-tuned version of microsoft/phi-2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0050
Rewards/chosen: -0.0996
Rewards/rejected: -19.3989
Rewards/accuracies: 0.9992
Rewards/margins: 19.2993
Logps/rejected: -644.0859
Logps/chosen: -275.4780
Logits/rejected: 0.2021
Logits/chosen: -0.3196

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 8
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 64
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6535	0.03	100	0.6404	0.0912	-0.0433	0.6936	0.1345	-450.5297	-273.5692	0.6266	0.1056
0.4293	0.07	200	0.4021	0.4957	-0.4032	0.8998	0.8989	-454.1287	-269.5248	0.6425	0.1325
0.2685	0.1	300	0.2464	0.8729	-1.0624	0.9386	1.9354	-460.7211	-265.7526	0.6700	0.1533
0.1918	0.14	400	0.1792	1.1981	-1.7756	0.9453	2.9737	-467.8528	-262.5003	0.6918	0.1719
0.1409	0.17	500	0.1423	1.4079	-2.4222	0.9503	3.8301	-474.3184	-260.4028	0.7139	0.1772
0.1437	0.2	600	0.1232	1.6054	-3.1009	0.9537	4.7063	-481.1060	-258.4279	0.7179	0.1571
0.0747	0.24	700	0.1057	1.7190	-3.9721	0.9613	5.6911	-489.8179	-257.2914	0.7179	0.1325
0.0848	0.27	800	0.0863	1.8540	-4.3289	0.9689	6.1829	-493.3858	-255.9415	0.7047	0.1024
0.0853	0.31	900	0.0744	1.8328	-5.0270	0.9722	6.8598	-500.3666	-256.1535	0.6746	0.0529
0.0561	0.34	1000	0.0639	1.7736	-5.4555	0.9747	7.2291	-504.6516	-256.7455	0.6517	0.0185
0.0657	0.37	1100	0.0546	1.7966	-5.6453	0.9840	7.4419	-506.5500	-256.5157	0.6738	0.0168
0.0445	0.41	1200	0.0499	1.7245	-6.0645	0.9823	7.7889	-510.7414	-257.2369	0.6757	0.0176
0.0498	0.44	1300	0.0459	1.6804	-6.6799	0.9848	8.3603	-516.8956	-257.6772	0.6719	0.0228
0.0554	0.48	1400	0.0549	1.5994	-6.8731	0.9848	8.4726	-518.8279	-258.4872	0.6684	0.0151
0.0368	0.51	1500	0.0365	1.3551	-7.8016	0.9865	9.1567	-528.1130	-260.9311	0.6464	-0.0223
0.0514	0.54	1600	0.0333	1.3431	-7.5785	0.9891	8.9215	-525.8817	-261.0511	0.6617	-0.0159
0.0365	0.58	1700	0.0310	1.0743	-8.8130	0.9933	9.8873	-538.2264	-263.7386	0.6354	-0.0234
0.0214	0.61	1800	0.0300	1.1310	-9.1633	0.9924	10.2942	-541.7292	-263.1721	0.6278	-0.0199
0.0321	0.65	1900	0.0267	0.9514	-9.9884	0.9924	10.9399	-549.9811	-264.9675	0.6044	-0.0268
0.0239	0.68	2000	0.0249	0.9624	-10.0608	0.9933	11.0232	-550.7047	-264.8574	0.5955	-0.0628
0.0484	0.71	2100	0.0222	0.9176	-11.0294	0.9941	11.9470	-560.3909	-265.3055	0.6043	-0.0338
0.0239	0.75	2200	0.0209	0.8247	-10.5070	0.9941	11.3317	-555.1663	-266.2345	0.5913	-0.0648
0.0219	0.78	2300	0.0221	0.7334	-10.5131	0.9924	11.2465	-555.2272	-267.1474	0.5942	-0.0808
0.0168	0.82	2400	0.0200	0.8909	-11.0818	0.9966	11.9727	-560.9146	-265.5724	0.6006	-0.0770
0.0135	0.85	2500	0.0187	0.7273	-11.3130	0.9958	12.0403	-563.2269	-267.2084	0.5948	-0.0711
0.0211	0.88	2600	0.0190	0.5390	-11.8492	0.9949	12.3883	-568.5890	-269.0913	0.5818	-0.0719
0.0209	0.92	2700	0.0170	0.4957	-12.3176	0.9949	12.8133	-573.2725	-269.5249	0.5537	-0.0912
0.0168	0.95	2800	0.0175	0.5850	-12.2737	0.9958	12.8587	-572.8340	-268.6321	0.5471	-0.0870
0.0129	0.99	2900	0.0171	0.5691	-12.7785	0.9949	13.3476	-577.8817	-268.7910	0.5441	-0.0810
0.0134	1.02	3000	0.0188	0.3713	-13.0254	0.9966	13.3966	-580.3504	-270.7691	0.5261	-0.1163
0.0127	1.05	3100	0.0155	0.1769	-14.3013	0.9958	14.4782	-593.1094	-272.7126	0.4928	-0.1129
0.0072	1.09	3200	0.0147	0.2474	-13.6499	0.9975	13.8973	-586.5952	-272.0074	0.4753	-0.1479
0.0068	1.12	3300	0.0179	0.2168	-14.6122	0.9958	14.8291	-596.2192	-272.3131	0.4722	-0.1430
0.0323	1.16	3400	0.0218	0.2536	-15.9479	0.9958	16.2015	-609.5762	-271.9459	0.4159	-0.1430
0.0232	1.19	3500	0.0171	0.4273	-14.2304	0.9958	14.6577	-592.4003	-270.2084	0.4598	-0.1410
0.0112	1.22	3600	0.0165	0.3634	-15.1069	0.9966	15.4704	-601.1661	-270.8472	0.4366	-0.1521
0.0136	1.26	3700	0.0176	0.2922	-14.4784	0.9966	14.7706	-594.8809	-271.5597	0.4553	-0.1679
0.0089	1.29	3800	0.0155	0.2349	-14.8970	0.9966	15.1319	-599.0669	-272.1331	0.4447	-0.1721
0.01	1.33	3900	0.0123	0.1309	-15.3620	0.9975	15.4930	-603.7170	-273.1724	0.4180	-0.1834
0.0055	1.36	4000	0.0132	0.0227	-16.5037	0.9975	16.5265	-615.1339	-274.2542	0.3836	-0.1899
0.004	1.39	4100	0.0122	0.2637	-15.2111	0.9975	15.4748	-602.2075	-271.8445	0.4087	-0.1870
0.0057	1.43	4200	0.0117	0.1593	-16.5411	0.9975	16.7003	-615.5074	-272.8891	0.4086	-0.1518
0.0041	1.46	4300	0.0113	-0.0815	-17.4813	0.9975	17.3998	-624.9092	-275.2963	0.3722	-0.1782
0.0039	1.5	4400	0.0115	0.2051	-17.4309	0.9983	17.6359	-624.4056	-272.4310	0.3445	-0.1975
0.0075	1.53	4500	0.0098	0.2810	-15.5168	0.9992	15.7978	-605.2650	-271.6719	0.4021	-0.1980
0.0034	1.56	4600	0.0123	0.2926	-15.4764	0.9992	15.7690	-604.8610	-271.5560	0.4225	-0.1766
0.0066	1.6	4700	0.0125	0.3192	-14.8325	0.9992	15.1517	-598.4222	-271.2900	0.4068	-0.2026
0.0079	1.63	4800	0.0124	0.1405	-15.1594	0.9992	15.2999	-601.6904	-273.0765	0.3942	-0.2089
0.0063	1.67	4900	0.0108	-0.1407	-17.4715	0.9975	17.3308	-624.8116	-275.8886	0.3294	-0.2140
0.0026	1.7	5000	0.0116	-0.0520	-17.9554	0.9992	17.9034	-629.6504	-275.0013	0.3361	-0.2062
0.0151	1.73	5100	0.0112	0.5727	-14.3495	0.9975	14.9221	-593.5914	-268.7550	0.4234	-0.1897
0.0049	1.77	5200	0.0083	0.2709	-15.7593	0.9975	16.0301	-607.6893	-271.7730	0.3788	-0.2054
0.0036	1.8	5300	0.0093	0.2336	-16.3789	0.9992	16.6125	-613.8856	-272.1456	0.3798	-0.1961
0.0076	1.84	5400	0.0086	0.2618	-16.1654	0.9992	16.4272	-611.7511	-271.8639	0.3844	-0.2090
0.0028	1.87	5500	0.0079	0.3444	-15.7817	0.9983	16.1261	-607.9135	-271.0378	0.3874	-0.2135
0.0102	1.9	5600	0.0081	0.3276	-15.7265	0.9983	16.0541	-607.3622	-271.2061	0.3713	-0.2243
0.0051	1.94	5700	0.0072	0.3979	-15.8670	0.9983	16.2649	-608.7670	-270.5030	0.3562	-0.2429
0.0068	1.97	5800	0.0069	0.2817	-16.7894	0.9992	17.0711	-617.9911	-271.6647	0.3329	-0.2424
0.0021	2.01	5900	0.0061	0.0725	-17.3567	0.9992	17.4292	-623.6634	-273.7564	0.3142	-0.2479
0.0031	2.04	6000	0.0066	-0.0179	-18.1905	0.9983	18.1726	-632.0013	-274.6606	0.2948	-0.2445
0.0032	2.07	6100	0.0064	0.0140	-17.8441	0.9992	17.8581	-628.5374	-274.3414	0.3043	-0.2424
0.0043	2.11	6200	0.0066	0.0216	-18.5824	0.9983	18.6041	-635.9210	-274.2653	0.2863	-0.2416
0.0033	2.14	6300	0.0070	0.0355	-18.3245	0.9983	18.3600	-633.3419	-274.1270	0.2783	-0.2691
0.0036	2.18	6400	0.0065	0.1473	-18.2292	0.9983	18.3765	-632.3882	-273.0083	0.2772	-0.2744
0.0023	2.21	6500	0.0064	0.0450	-17.9052	0.9992	17.9502	-629.1483	-274.0311	0.2750	-0.2756
0.0025	2.24	6600	0.0061	-0.0642	-17.4265	0.9983	17.3623	-624.3619	-275.1237	0.2773	-0.2881
0.0008	2.28	6700	0.0058	0.0951	-17.3710	0.9992	17.4661	-623.8071	-273.5309	0.2703	-0.2891
0.002	2.31	6800	0.0057	-0.0533	-18.2744	0.9992	18.2212	-632.8410	-275.0142	0.2544	-0.2823
0.0052	2.35	6900	0.0073	0.0152	-18.6276	0.9983	18.6427	-636.3723	-274.3301	0.2365	-0.2928
0.0029	2.38	7000	0.0056	0.0321	-18.4600	0.9992	18.4921	-634.6968	-274.1606	0.2366	-0.2889
0.0013	2.41	7100	0.0058	-0.1304	-19.1740	0.9983	19.0436	-641.8367	-275.7859	0.2306	-0.2898
0.0011	2.45	7200	0.0059	-0.0977	-19.2699	0.9992	19.1722	-642.7958	-275.4590	0.2227	-0.2916
0.0058	2.48	7300	0.0059	-0.0938	-18.8625	0.9992	18.7687	-638.7216	-275.4195	0.2314	-0.2945
0.004	2.52	7400	0.0067	-0.1457	-19.2264	0.9992	19.0807	-642.3610	-275.9389	0.2194	-0.2959
0.0042	2.55	7500	0.0056	-0.0100	-18.6009	0.9992	18.5909	-636.1058	-274.5812	0.2398	-0.3003
0.0033	2.58	7600	0.0057	-0.1484	-19.3114	0.9992	19.1630	-643.2104	-275.9655	0.2215	-0.2965
0.0012	2.62	7700	0.0051	-0.1574	-19.2398	0.9992	19.0824	-642.4946	-276.0552	0.2210	-0.3052
0.002	2.65	7800	0.0052	-0.1220	-19.5223	0.9992	19.4004	-645.3201	-275.7011	0.2102	-0.3165
0.0053	2.69	7900	0.0051	-0.1063	-19.6889	0.9992	19.5826	-646.9856	-275.5444	0.2106	-0.3074
0.0013	2.72	8000	0.0056	-0.1357	-19.8339	0.9992	19.6982	-648.4356	-275.8386	0.2029	-0.3129
0.002	2.75	8100	0.0050	-0.1024	-19.6476	0.9992	19.5452	-646.5726	-275.5056	0.1993	-0.3168
0.0016	2.79	8200	0.0052	-0.0378	-19.4058	0.9992	19.3680	-644.1544	-274.8596	0.2152	-0.3120
0.0015	2.82	8300	0.0049	-0.0549	-19.2544	0.9992	19.1994	-642.6404	-275.0310	0.2093	-0.3168
0.003	2.86	8400	0.0051	-0.1218	-19.2614	0.9983	19.1396	-642.7106	-275.7000	0.2057	-0.3182
0.0019	2.89	8500	0.0050	-0.1057	-19.3041	0.9992	19.1983	-643.1373	-275.5387	0.2077	-0.3151
0.0039	2.92	8600	0.0052	-0.0863	-19.2826	0.9992	19.1963	-642.9229	-275.3445	0.2057	-0.3208
0.0042	2.96	8700	0.0048	-0.1111	-19.3837	0.9992	19.2726	-643.9339	-275.5926	0.2008	-0.3215
0.0023	2.99	8800	0.0050	-0.1085	-19.4076	0.9992	19.2990	-644.1723	-275.5668	0.1985	-0.3228

Framework versions

Transformers 4.38.0.dev0
Pytorch 2.1.1+cu121
Datasets 2.14.6
Tokenizers 0.15.1

Task: Text Generation
Library: transformers
Type: phi
License: mit
Created: Jan 25, 2024
Updated: Jan 26, 2024