phi-3-mini-LoRA

KrithikV/phi-3-mini-LoRA

phi-3-mini-LoRA is a machine learning model from KrithikV. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

16% of all-time downloads

All-time downloads

Public

Repo size

71.8 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors35.7 MB · 94%

At a glance

Library: peft
License: mit
Access: Public
Created: Jul 29, 2024
Updated: Jul 30, 2024
SHA: dbc2bad1

Base models

microsoft/Phi-3-mini-4k-instruct

Model card

From the Hugging Face model README

phi-3-mini-LoRA

This model is a fine-tuned version of microsoft/Phi-3-mini-4k-instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.8445

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.0374	0.0329	100	1.0194
0.9715	0.0658	200	0.9314
0.9106	0.0987	300	0.8969
0.888	0.1316	400	0.8869
0.8902	0.1645	500	0.8813
0.8826	0.1974	600	0.8777
0.8763	0.2303	700	0.8745
0.8728	0.2632	800	0.8723
0.8707	0.2961	900	0.8701
0.8702	0.3289	1000	0.8684
0.8631	0.3618	1100	0.8664
0.8623	0.3947	1200	0.8647
0.8655	0.4276	1300	0.8624
0.863	0.4605	1400	0.8602
0.858	0.4934	1500	0.8586
0.859	0.5263	1600	0.8578
0.8527	0.5592	1700	0.8569
0.8587	0.5921	1800	0.8563
0.8551	0.625	1900	0.8557
0.8548	0.6579	2000	0.8550
0.8515	0.6908	2100	0.8546
0.8531	0.7237	2200	0.8542
0.8567	0.7566	2300	0.8535
0.8589	0.7895	2400	0.8532
0.8547	0.8224	2500	0.8529
0.8537	0.8553	2600	0.8525
0.85	0.8882	2700	0.8521
0.8518	0.9211	2800	0.8519
0.8456	0.9539	2900	0.8515
0.8585	0.9868	3000	0.8512
0.849	1.0197	3100	0.8509
0.8549	1.0526	3200	0.8507
0.8502	1.0855	3300	0.8504
0.8504	1.1184	3400	0.8502
0.8488	1.1513	3500	0.8500
0.8504	1.1842	3600	0.8497
0.8465	1.2171	3700	0.8495
0.8471	1.25	3800	0.8494
0.8467	1.2829	3900	0.8491
0.8439	1.3158	4000	0.8489
0.8467	1.3487	4100	0.8487
0.8461	1.3816	4200	0.8485
0.8525	1.4145	4300	0.8483
0.8426	1.4474	4400	0.8481
0.8479	1.4803	4500	0.8480
0.853	1.5132	4600	0.8478
0.8432	1.5461	4700	0.8477
0.8416	1.5789	4800	0.8475
0.8527	1.6118	4900	0.8474
0.849	1.6447	5000	0.8472
0.8446	1.6776	5100	0.8471
0.8427	1.7105	5200	0.8469
0.8464	1.7434	5300	0.8468
0.8444	1.7763	5400	0.8466
0.8479	1.8092	5500	0.8465
0.8452	1.8421	5600	0.8465
0.8387	1.875	5700	0.8466
0.845	1.9079	5800	0.8463
0.8402	1.9408	5900	0.8461
0.8459	1.9737	6000	0.8460
0.8431	2.0066	6100	0.8460
0.8395	2.0395	6200	0.8459
0.8395	2.0724	6300	0.8458
0.8457	2.1053	6400	0.8457
0.8438	2.1382	6500	0.8457
0.8411	2.1711	6600	0.8456
0.8386	2.2039	6700	0.8456
0.8393	2.2368	6800	0.8454
0.8406	2.2697	6900	0.8454
0.8386	2.3026	7000	0.8453
0.8456	2.3355	7100	0.8453
0.8424	2.3684	7200	0.8452
0.8437	2.4013	7300	0.8451
0.8426	2.4342	7400	0.8451
0.8393	2.4671	7500	0.8450
0.8398	2.5	7600	0.8450
0.8434	2.5329	7700	0.8449
0.8456	2.5658	7800	0.8449
0.8393	2.5987	7900	0.8449
0.8401	2.6316	8000	0.8448
0.838	2.6645	8100	0.8448
0.8432	2.6974	8200	0.8447
0.8471	2.7303	8300	0.8447
0.8435	2.7632	8400	0.8446
0.8441	2.7961	8500	0.8446
0.8399	2.8289	8600	0.8446
0.8391	2.8618	8700	0.8446
0.8432	2.8947	8800	0.8446
0.8459	2.9276	8900	0.8446
0.8446	2.9605	9000	0.8445
0.8412	2.9934	9100	0.8445

Framework versions

PEFT 0.12.0
Transformers 4.43.3
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1

Library: peft
License: mit
Created: Jul 29, 2024
Updated: Jul 30, 2024