UTI_L3_500steps_1e7rate_01beta_CSFTDPO

tsavage68/UTI_L3_500steps_1e7rate_01beta_CSFTDPO

UTI_L3_500steps_1e7rate_01beta_CSFTDPO is a text generation model from tsavage68. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

28% of all-time downloads

All-time downloads

Public

Parameters

16.1 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors32.1 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: llama3
Model type: llama
Access: Public
Created: Jun 5, 2024
Updated: Jun 5, 2024
SHA: 66e544ef

Try a prompt

Base models

tsavage68/UTI_L3_1000steps_1e5rate_SFT

Model card

From the Hugging Face model README

UTI_L3_500steps_1e7rate_01beta_CSFTDPO

This model is a fine-tuned version of tsavage68/UTI_L3_1000steps_1e5rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.2926
Rewards/chosen: 0.2081
Rewards/rejected: -1.1090
Rewards/accuracies: 0.9900
Rewards/margins: 1.3171
Logps/rejected: -74.2848
Logps/chosen: -30.3985
Logits/rejected: -1.3260
Logits/chosen: -1.3110

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-07
train_batch_size: 2
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6922	0.3333	25	0.6925	-0.0003	-0.0016	0.5400	0.0013	-63.2107	-32.4819	-1.3229	-1.3078
0.6843	0.6667	50	0.6806	0.0044	-0.0210	0.8100	0.0254	-63.4048	-32.4353	-1.3232	-1.3080
0.6624	1.0	75	0.6486	0.0165	-0.0759	0.9400	0.0923	-63.9532	-32.3144	-1.3233	-1.3082
0.5995	1.3333	100	0.5895	0.0366	-0.1897	0.9700	0.2262	-65.0915	-32.1134	-1.3238	-1.3089
0.5302	1.6667	125	0.5192	0.0674	-0.3418	0.9900	0.4092	-66.6123	-31.8046	-1.3240	-1.3091
0.4705	2.0	150	0.4616	0.0967	-0.4861	0.9900	0.5828	-68.0561	-31.5123	-1.3241	-1.3093
0.3935	2.3333	175	0.4138	0.1216	-0.6328	0.9900	0.7545	-69.5230	-31.2626	-1.3245	-1.3097
0.3748	2.6667	200	0.3806	0.1427	-0.7384	0.9900	0.8811	-70.5788	-31.0524	-1.3250	-1.3102
0.3436	3.0	225	0.3504	0.1630	-0.8502	0.9900	1.0131	-71.6963	-30.8493	-1.3251	-1.3103
0.3577	3.3333	250	0.3324	0.1749	-0.9257	0.9900	1.1006	-72.4519	-30.7302	-1.3257	-1.3108
0.2912	3.6667	275	0.3164	0.1877	-0.9963	0.9900	1.1839	-73.1575	-30.6024	-1.3257	-1.3108
0.3042	4.0	300	0.3063	0.1951	-1.0428	0.9900	1.2379	-73.6230	-30.5284	-1.3258	-1.3108
0.2635	4.3333	325	0.2996	0.2024	-1.0747	0.9900	1.2771	-73.9418	-30.4550	-1.3258	-1.3108
0.2766	4.6667	350	0.2958	0.2048	-1.0938	0.9900	1.2986	-74.1325	-30.4309	-1.3259	-1.3108
0.2949	5.0	375	0.2936	0.2074	-1.1029	0.9900	1.3102	-74.2233	-30.4053	-1.3259	-1.3109
0.2943	5.3333	400	0.2930	0.2070	-1.1083	0.9900	1.3153	-74.2776	-30.4092	-1.3259	-1.3109
0.2709	5.6667	425	0.2922	0.2083	-1.1091	0.9900	1.3174	-74.2857	-30.3961	-1.3260	-1.3110
0.2615	6.0	450	0.2924	0.2081	-1.1078	0.9900	1.3159	-74.2726	-30.3975	-1.3260	-1.3109
0.256	6.3333	475	0.2926	0.2081	-1.1090	0.9900	1.3171	-74.2848	-30.3985	-1.3260	-1.3110
0.2969	6.6667	500	0.2926	0.2081	-1.1090	0.9900	1.3171	-74.2848	-30.3985	-1.3260	-1.3110

Framework versions

Transformers 4.41.2
Pytorch 2.0.0+cu117
Datasets 2.19.2
Tokenizers 0.19.1

Task: Text Generation
Library: transformers
Type: llama
License: llama3
Created: Jun 5, 2024
Updated: Jun 5, 2024