fine-tuning-Phi2-with-webglm-qa-with-lora_7

Gunslinger3D/fine-tuning-Phi2-with-webglm-qa-with-lora_7

fine-tuning-Phi2-with-webglm-qa-with-lora_7 is a machine learning model from Gunslinger3D. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

2% of all-time downloads

All-time downloads

111

Public

Repo size

142 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors189 MB · 79%

At a glance

Library: peft
License: mit
Access: Public
Created: Mar 26, 2024
Updated: Mar 26, 2024
SHA: 66dc4a43

Base models

microsoft/phi-2

Model card

From the Hugging Face model README

fine-tuning-Phi2-with-webglm-qa-with-lora_7

This model is a fine-tuned version of microsoft/phi-2 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.0950

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 2
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 5
total_train_batch_size: 10
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 60
training_steps: 1000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
7.3505	0.31	20	6.2863
4.0914	0.63	40	0.9255
0.6517	0.94	60	0.5762
0.4621	1.26	80	0.4062
0.3128	1.57	100	0.3056
0.2536	1.89	120	0.2604
0.2227	2.2	140	0.2247
0.1901	2.52	160	0.2041
0.176	2.83	180	0.1812
0.1453	3.14	200	0.1683
0.1557	3.46	220	0.1592
0.1441	3.77	240	0.1488
0.1282	4.09	260	0.1430
0.1215	4.4	280	0.1348
0.1217	4.72	300	0.1323
0.117	5.03	320	0.1271
0.109	5.35	340	0.1255
0.1094	5.66	360	0.1210
0.1057	5.97	380	0.1175
0.0937	6.29	400	0.1158
0.0942	6.6	420	0.1159
0.1007	6.92	440	0.1125
0.0876	7.23	460	0.1119
0.0894	7.55	480	0.1099
0.0827	7.86	500	0.1072
0.0894	8.18	520	0.1069
0.0805	8.49	540	0.1075
0.0782	8.81	560	0.1043
0.0881	9.12	580	0.1034
0.0839	9.43	600	0.1015
0.0694	9.75	620	0.1000
0.068	10.06	640	0.1007
0.072	10.38	660	0.0994
0.0709	10.69	680	0.0985
0.0712	11.01	700	0.0986
0.0673	11.32	720	0.0999
0.0669	11.64	740	0.0974
0.0706	11.95	760	0.0981
0.0641	12.26	780	0.0969
0.0652	12.58	800	0.0964
0.0668	12.89	820	0.0962
0.0617	13.21	840	0.0972
0.0628	13.52	860	0.0960
0.0637	13.84	880	0.0949
0.0633	14.15	900	0.0951
0.0577	14.47	920	0.0953
0.0646	14.78	940	0.0947
0.06	15.09	960	0.0946
0.0584	15.41	980	0.0949
0.0638	15.72	1000	0.0950

Framework versions

PEFT 0.7.1
Transformers 4.36.2
Pytorch 2.0.0
Datasets 2.15.0
Tokenizers 0.15.0

Library: peft
License: mit
Created: Mar 26, 2024
Updated: Mar 26, 2024