zephyr-7b-dpo-full-hydrox-safe

yihang7/zephyr-7b-dpo-full-hydrox-safe

zephyr-7b-dpo-full-hydrox-safe is a text generation model from yihang7. Use it when you need the model to write or continue text. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

6% of all-time downloads

All-time downloads

Public

Parameters

7.2B

14.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors14.5 GB · 100%

At a glance

Task: Text Generation
Library: transformers
Model type: mistral
Access: Public
Created: Jan 12, 2024
Updated: Jan 13, 2024
SHA: 7ce1e998

Try a prompt

Model card

From the Hugging Face model README

zephyr-7b-dpo-full-hydrox-safe

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0018
Rewards/chosen: 1.5348
Rewards/rejected: -41.0322
Rewards/accuracies: 0.9992
Rewards/margins: 42.5670
Logps/rejected: -855.1025
Logps/chosen: -270.7936
Logits/rejected: -2.8721
Logits/chosen: -3.0906

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 8
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 64
total_eval_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.201	0.03	100	0.1790	0.8766	-1.8347	0.9571	2.7113	-463.1273	-277.3755	-2.9298	-3.0062
0.1124	0.07	200	0.0931	1.4748	-3.5536	0.9764	5.0284	-480.3171	-271.3938	-2.9144	-2.9984
0.0657	0.1	300	0.1207	1.7581	-4.7603	0.9806	6.5185	-492.3842	-268.5603	-2.9028	-2.9908
0.0676	0.14	400	0.0446	1.3392	-8.0193	0.9857	9.3585	-524.9738	-272.75	-2.9171	-3.0132
0.0477	0.17	500	0.0581	1.3777	-9.9106	0.9882	11.2883	-543.8868	-272.3650	-2.8810	-2.9695
0.0404	0.2	600	0.0845	0.3594	-12.1256	0.9840	12.4850	-566.0372	-282.5481	-2.8164	-2.9235
0.0122	0.24	700	0.0831	0.4374	-19.3178	0.9848	19.7552	-637.9587	-281.7681	-2.7024	-2.8110
0.0183	0.27	800	0.0775	-0.0400	-18.4153	0.9891	18.3753	-628.9333	-286.5414	-2.8078	-2.9358
0.0555	0.31	900	0.0589	0.4664	-23.4223	0.9924	23.8887	-679.0039	-281.4781	-2.8207	-2.9406
0.0376	0.34	1000	0.0552	0.6445	-26.2370	0.9874	26.8814	-707.1502	-279.6971	-2.7989	-2.9241
0.0438	0.37	1100	0.0685	1.7090	-23.4352	0.9874	25.1442	-679.1324	-269.0518	-2.8438	-3.0501
0.074	0.41	1200	0.0715	1.4961	-24.8055	0.9882	26.3016	-692.8359	-271.1810	-2.7269	-2.8728
0.1029	0.44	1300	0.1098	0.9065	-28.0158	0.9882	28.9223	-724.9388	-277.0765	-2.7411	-2.9059
0.0271	0.48	1400	0.0346	0.7923	-26.9632	0.9907	27.7555	-714.4124	-278.2186	-2.7555	-2.9286
0.0219	0.51	1500	0.0405	0.5053	-27.0039	0.9933	27.5092	-714.8198	-281.0890	-2.7955	-2.9553
0.0214	0.54	1600	0.0293	0.7598	-22.6033	0.9941	23.3631	-670.8132	-278.5433	-2.7650	-2.9211
0.0221	0.58	1700	0.0206	0.6463	-25.7837	0.9949	26.4300	-702.6176	-279.6782	-2.7340	-2.9148
0.0339	0.61	1800	0.0204	1.2579	-22.2971	0.9966	23.5550	-667.7516	-273.5627	-2.7337	-2.8868
0.0662	0.65	1900	0.0290	0.9343	-30.7737	0.9975	31.7081	-752.5181	-276.7985	-2.7130	-2.8608
0.1032	0.68	2000	0.0358	1.6616	-22.3776	0.9975	24.0392	-668.5566	-269.5255	-2.7454	-2.8772
0.0059	0.71	2100	0.0148	0.9711	-28.7176	0.9949	29.6887	-731.9562	-276.4307	-2.6758	-2.8253
0.0096	0.75	2200	0.0282	0.9406	-25.7753	0.9949	26.7158	-702.5333	-276.7361	-2.9417	-3.1166
0.0117	0.78	2300	0.0489	-0.3402	-33.3639	0.9924	33.0238	-778.4199	-289.5432	-3.0059	-3.1712
0.0034	0.82	2400	0.0136	1.4644	-27.4436	0.9966	28.9079	-719.2164	-271.4980	-2.6572	-2.7758
0.0448	0.85	2500	0.0122	-0.7904	-29.3707	0.9983	28.5803	-738.4877	-294.0459	-2.7608	-2.8870
0.0606	0.88	2600	0.0723	0.6824	-40.6246	0.9949	41.3070	-851.0272	-279.3180	-2.6804	-2.8281
0.0172	0.92	2700	0.0151	1.7774	-28.1840	0.9958	29.9614	-726.6205	-268.3677	-2.7720	-2.9085
0.0206	0.95	2800	0.0105	0.0966	-39.5676	0.9975	39.6643	-840.4568	-285.1752	-2.8397	-2.9803
0.0093	0.99	2900	0.0165	0.6995	-33.4341	0.9966	34.1336	-779.1221	-279.1465	-2.8086	-2.9511
0.0008	1.02	3000	0.0101	-0.0075	-36.2959	0.9983	36.2884	-807.7395	-286.2166	-2.9213	-3.0591
0.0015	1.05	3100	0.0091	-0.2430	-38.1496	0.9966	37.9066	-826.2767	-288.5718	-2.8160	-2.9881
0.001	1.09	3200	0.0069	0.7306	-37.6657	0.9966	38.3964	-821.4379	-278.8353	-2.8287	-2.9686
0.0447	1.12	3300	0.0148	0.4641	-39.1245	0.9975	39.5886	-836.0254	-281.5004	-2.8502	-2.9821
0.0015	1.16	3400	0.0145	1.2867	-34.9331	0.9983	36.2198	-794.1116	-273.2742	-2.8645	-2.9793
0.0069	1.19	3500	0.0120	1.0980	-33.3648	0.9983	34.4628	-778.4290	-275.1619	-2.8298	-2.9435
0.0006	1.22	3600	0.0144	0.4168	-38.9632	0.9975	39.3800	-834.4130	-281.9736	-2.8061	-2.9050
0.0532	1.26	3700	0.0163	1.2493	-35.1549	0.9983	36.4042	-796.3295	-273.6483	-3.0117	-3.1271
0.0041	1.29	3800	0.0138	0.6365	-36.2498	0.9966	36.8863	-807.2789	-279.7767	-2.9031	-3.0283
0.0043	1.33	3900	0.0095	0.8546	-36.5756	0.9992	37.4302	-810.5366	-277.5952	-2.9713	-3.0945
0.0016	1.36	4000	0.0098	0.4731	-41.9454	0.9992	42.4186	-864.2352	-281.4105	-3.0390	-3.1458
0.0018	1.39	4100	0.0103	1.1650	-31.5836	0.9992	32.7487	-760.6172	-274.4915	-3.0057	-3.1525
0.0015	1.43	4200	0.0097	0.7472	-35.6321	0.9992	36.3793	-801.1016	-278.6696	-2.9895	-3.1466
0.0078	1.46	4300	0.0171	0.4428	-47.4943	0.9983	47.9370	-919.7234	-281.7140	-3.0189	-3.1577
0.0006	1.5	4400	0.0117	2.3601	-36.7779	0.9992	39.1379	-812.5596	-262.5412	-2.9504	-3.0998
0.0021	1.53	4500	0.0096	1.8317	-33.0472	0.9983	34.8789	-775.2529	-267.8244	-2.8650	-3.0361
0.0022	1.56	4600	0.0124	1.8244	-35.2193	0.9983	37.0437	-796.9737	-267.8973	-2.8983	-3.0580
0.0025	1.6	4700	0.0120	1.4261	-35.4094	0.9983	36.8355	-798.8742	-271.8802	-2.8938	-3.0468
0.0154	1.63	4800	0.0070	0.7832	-39.4533	0.9983	40.2365	-839.3135	-278.3098	-2.9491	-3.1257
0.0025	1.67	4900	0.0070	0.5157	-40.3519	0.9983	40.8676	-848.2994	-280.9843	-2.8537	-3.0217
0.0022	1.7	5000	0.0060	1.8378	-36.1887	0.9983	38.0265	-806.6680	-267.7638	-2.8008	-2.9944
0.0116	1.73	5100	0.0067	1.6571	-30.2966	0.9983	31.9537	-747.7468	-269.5709	-2.8245	-3.0273
0.0027	1.77	5200	0.0056	1.8559	-28.9170	0.9983	30.7729	-733.9506	-267.5823	-2.8308	-3.0333
0.0008	1.8	5300	0.0120	0.1806	-39.8991	0.9983	40.0797	-843.7715	-284.3351	-3.0642	-3.2708
0.0008	1.84	5400	0.0042	1.8857	-32.3213	0.9983	34.2070	-767.9935	-267.2849	-2.9745	-3.1835
0.0419	1.87	5500	0.0049	0.4195	-42.5787	0.9983	42.9982	-870.5677	-281.9464	-2.9390	-3.1435
0.0055	1.9	5600	0.0039	0.8564	-39.5863	0.9966	40.4428	-840.6442	-277.5772	-2.9517	-3.1465
0.0016	1.94	5700	0.0039	1.5280	-38.4427	0.9983	39.9707	-829.2079	-270.8615	-2.9334	-3.1210
0.0015	1.97	5800	0.0031	2.5689	-32.1821	0.9992	34.7510	-766.6016	-260.4530	-2.9266	-3.1170
0.0022	2.01	5900	0.0154	1.5272	-38.5973	0.9983	40.1246	-830.7538	-270.8692	-2.9905	-3.1795
0.0001	2.04	6000	0.0153	1.4304	-38.7250	0.9983	40.1554	-832.0308	-271.8378	-3.0212	-3.2078
0.0004	2.07	6100	0.0151	0.9974	-40.5361	0.9983	41.5335	-850.1418	-276.1674	-3.0092	-3.1920
0.0032	2.11	6200	0.0143	2.2246	-35.4872	0.9983	37.7117	-799.6525	-263.8961	-3.0169	-3.2017
0.0026	2.14	6300	0.0138	2.1149	-36.0655	0.9983	38.1804	-805.4358	-264.9924	-3.0121	-3.1974
0.0052	2.18	6400	0.0093	2.0832	-38.0872	0.9983	40.1704	-825.6531	-265.3097	-2.9119	-3.1190
0.0002	2.21	6500	0.0031	2.4808	-33.4480	0.9992	35.9287	-779.2603	-261.3340	-2.9056	-3.1235
0.0014	2.24	6600	0.0029	2.1622	-35.1487	0.9992	37.3110	-796.2680	-264.5192	-2.9322	-3.1505
0.0001	2.28	6700	0.0029	2.6987	-34.3087	0.9992	37.0074	-787.8676	-259.1545	-2.9528	-3.1683
0.0004	2.31	6800	0.0028	2.2809	-36.5033	0.9992	38.7842	-809.8135	-263.3324	-2.9377	-3.1456
0.0268	2.35	6900	0.0028	1.9206	-37.6915	0.9983	39.6121	-821.6957	-266.9360	-2.9110	-3.1199
0.0012	2.38	7000	0.0026	2.0357	-34.8535	0.9992	36.8892	-793.3152	-265.7842	-2.9165	-3.1204
0.0002	2.41	7100	0.0031	0.4887	-42.0925	0.9983	42.5812	-865.7061	-281.2549	-2.8696	-3.0815
0.0006	2.45	7200	0.0027	1.6025	-38.2160	0.9992	39.8185	-826.9409	-270.1166	-2.8921	-3.1009
0.0037	2.48	7300	0.0027	1.2715	-42.6264	0.9992	43.8978	-871.0444	-273.4270	-2.8870	-3.0973
0.0023	2.52	7400	0.0026	1.3232	-42.5318	0.9992	43.8549	-870.0981	-272.9100	-2.8912	-3.1000
0.0012	2.55	7500	0.0026	1.4663	-41.9917	0.9992	43.4581	-864.6981	-271.4783	-2.8990	-3.1084
0.0012	2.58	7600	0.0025	1.2190	-42.9826	0.9992	44.2016	-874.6072	-273.9521	-2.8991	-3.1070
0.0001	2.62	7700	0.0026	1.7469	-40.1663	0.9992	41.9132	-846.4437	-268.6728	-2.8615	-3.0744
0.0001	2.65	7800	0.0026	1.7291	-40.4524	0.9992	42.1815	-849.3045	-268.8504	-2.8619	-3.0751
0.003	2.69	7900	0.0018	1.8715	-39.4482	0.9992	41.3197	-839.2630	-267.4267	-2.8391	-3.0504
0.0001	2.72	8000	0.0018	1.4707	-42.0026	0.9992	43.4733	-864.8069	-271.4347	-2.8587	-3.0725
0.0001	2.75	8100	0.0018	1.7032	-41.2255	0.9992	42.9287	-857.0361	-269.1100	-2.8611	-3.0771
0.0001	2.79	8200	0.0018	1.7053	-41.5715	0.9992	43.2768	-860.4957	-269.0886	-2.8608	-3.0769
0.0	2.82	8300	0.0017	1.5198	-41.7233	0.9992	43.2431	-862.0136	-270.9432	-2.8563	-3.0742
0.0	2.86	8400	0.0018	1.7544	-40.1153	0.9992	41.8696	-845.9333	-268.5981	-2.8694	-3.0853
0.0003	2.89	8500	0.0018	1.6997	-40.2116	0.9992	41.9113	-846.8971	-269.1451	-2.8782	-3.0951
0.0071	2.92	8600	0.0018	1.5913	-40.7278	0.9992	42.3191	-852.0585	-270.2284	-2.8728	-3.0908
0.0013	2.96	8700	0.0018	1.5263	-41.0232	0.9992	42.5495	-855.0129	-270.8784	-2.8716	-3.0900
0.0003	2.99	8800	0.0019	1.5305	-41.0444	0.9992	42.5748	-855.2242	-270.8367	-2.8724	-3.0907

Framework versions

Transformers 4.35.0
Pytorch 2.1.1+cu121
Datasets 2.14.6
Tokenizers 0.14.1

Task: Text Generation
Library: transformers
Type: mistral
Created: Jan 12, 2024
Updated: Jan 13, 2024