mistralit2_1000_STEPS_5e7_rate_03_beta_DPO

tsavage68/mistralit2_1000_STEPS_5e7_rate_03_beta_DPO

mistralit2_1000_STEPS_5e7_rate_03_beta_DPO is a text generation model from tsavage68. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

7% of all-time downloads

All-time downloads

Public

Parameters

7.2B

14.5 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors29 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: apache-2.0
Model type: mistral
Access: Public
Created: Mar 11, 2024
Updated: Mar 11, 2024
SHA: c95a15ec

Try a prompt

Base models

mistralai/Mistral-7B-Instruct-v0.2

Model card

From the Hugging Face model README

mistralit2_1000_STEPS_5e7_rate_03_beta_DPO

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 1.0554
Rewards/chosen: -4.6458
Rewards/rejected: -7.9897
Rewards/accuracies: 0.6593
Rewards/margins: 3.3439
Logps/rejected: -55.2048
Logps/chosen: -38.8718
Logits/rejected: -2.6256
Logits/chosen: -2.6266

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-07
train_batch_size: 4
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6053	0.1	50	0.6740	-0.3080	-0.4763	0.5429	0.1682	-30.1599	-24.4126	-2.8583	-2.8587
0.6455	0.2	100	0.6888	-1.9296	-2.8473	0.6110	0.9176	-38.0633	-29.8180	-2.7011	-2.7016
0.6646	0.29	150	0.8842	-4.4677	-5.7716	0.5956	1.3039	-47.8112	-38.2782	-2.7068	-2.7076
0.8576	0.39	200	0.8269	0.2095	-0.3290	0.5341	0.5385	-29.6690	-22.6876	-2.8074	-2.8077
0.9282	0.49	250	0.8715	-3.3030	-4.1864	0.5758	0.8834	-42.5272	-34.3958	-2.8320	-2.8326
0.883	0.59	300	0.8491	-1.6930	-2.7293	0.5846	1.0364	-37.6702	-29.0290	-2.8023	-2.8028
0.7641	0.68	350	0.8305	-0.5284	-1.4934	0.5868	0.9650	-33.5504	-25.1471	-2.8008	-2.8013
0.8485	0.78	400	0.8168	-1.8042	-3.2662	0.6286	1.4620	-39.4597	-29.3999	-2.8978	-2.8983
0.6637	0.88	450	0.9089	-4.1779	-5.6349	0.6220	1.4570	-47.3556	-37.3123	-2.7996	-2.8003
0.8293	0.98	500	0.7790	-1.7260	-3.1768	0.6242	1.4508	-39.1617	-29.1392	-2.7937	-2.7943
0.1061	1.07	550	0.8642	-2.6748	-4.9677	0.6659	2.2929	-45.1314	-32.3019	-2.7609	-2.7616
0.1183	1.17	600	1.0052	-4.2792	-7.1691	0.6527	2.8899	-52.4695	-37.6498	-2.6760	-2.6769
0.3423	1.27	650	1.0032	-4.1972	-7.1444	0.6571	2.9472	-52.3871	-37.3765	-2.6563	-2.6572
0.3015	1.37	700	1.0111	-4.0263	-7.1542	0.6549	3.1280	-52.4198	-36.8067	-2.6518	-2.6526
0.0814	1.46	750	1.0416	-4.3351	-7.5972	0.6484	3.2621	-53.8964	-37.8360	-2.6335	-2.6344
0.1279	1.56	800	1.0511	-4.6097	-7.9321	0.6505	3.3224	-55.0127	-38.7514	-2.6277	-2.6287
0.1507	1.66	850	1.0478	-4.6393	-7.9834	0.6484	3.3441	-55.1838	-38.8501	-2.6262	-2.6272
0.2148	1.76	900	1.0515	-4.6439	-7.9924	0.6527	3.3485	-55.2139	-38.8655	-2.6260	-2.6270
0.2291	1.86	950	1.0554	-4.6452	-7.9877	0.6505	3.3425	-55.1980	-38.8697	-2.6257	-2.6267
0.13	1.95	1000	1.0554	-4.6458	-7.9897	0.6593	3.3439	-55.2048	-38.8718	-2.6256	-2.6266

Framework versions

Transformers 4.38.2
Pytorch 2.0.0+cu117
Datasets 2.18.0
Tokenizers 0.15.2

Task: Text Generation
Library: transformers
Type: mistral
License: apache-2.0
Created: Mar 11, 2024
Updated: Mar 11, 2024