GenAI-task-2-ModelD-DS

Katochh/GenAI-task-2-ModelD-DS

GenAI-task-2-ModelD-DS is a machine learning model from Katochh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

16% of all-time downloads

All-time downloads

Public

Repo size

1.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors25.2 MB · 88%

At a glance

Library: peft
License: apache-2.0
Access: Public
Created: Apr 28, 2024
Updated: Apr 28, 2024
SHA: f41f5d62

Base models

petals-team/falcon-rw-1b

Model card

From the Hugging Face model README

GenAI-task-2-ModelD-DS

This model is a fine-tuned version of petals-team/falcon-rw-1b on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.8551

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 2
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.01
num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
1.538	0.0316	20	1.4898
2.1944	0.0631	40	1.4676
2.2479	0.0947	60	1.4396
1.7654	0.1263	80	1.3892
2.1763	0.1579	100	1.3834
1.3644	0.1894	120	1.3282
1.6781	0.2210	140	1.3058
1.7429	0.2526	160	1.2880
1.37	0.2841	180	1.2483
1.8196	0.3157	200	1.2511
1.223	0.3473	220	1.2120
1.5357	0.3788	240	1.2171
1.6471	0.4104	260	1.1906
1.271	0.4420	280	1.1818
1.7222	0.4736	300	1.1788
1.2022	0.5051	320	1.1170
1.4455	0.5367	340	1.1633
1.7014	0.5683	360	1.1011
1.1309	0.5998	380	1.0815
1.6978	0.6314	400	1.0966
1.0796	0.6630	420	1.0325
1.4504	0.6946	440	1.0429
1.4698	0.7261	460	1.0216
1.0858	0.7577	480	1.0031
1.4275	0.7893	500	1.0115
0.9607	0.8208	520	0.9771
1.2579	0.8524	540	0.9792
1.3363	0.8840	560	0.9608
1.0551	0.9155	580	0.9471
1.531	0.9471	600	0.9530
0.9776	0.9787	620	0.9321
1.374	1.0103	640	0.9257
0.9688	1.0418	660	0.9217
1.464	1.0734	680	0.9278
1.0608	1.1050	700	0.9040
1.0711	1.1365	720	0.9017
1.2806	1.1681	740	0.8954
0.9129	1.1997	760	0.8877
1.2161	1.2313	780	0.8907
1.0221	1.2628	800	0.8794
1.1306	1.2944	820	0.8782
1.3235	1.3260	840	0.8768
0.9663	1.3575	860	0.8711
1.3124	1.3891	880	0.8716
1.0169	1.4207	900	0.8663
1.1686	1.4522	920	0.8658
1.2976	1.4838	940	0.8656
0.8896	1.5154	960	0.8620
1.3252	1.5470	980	0.8623
1.0821	1.5785	1000	0.8601
1.1595	1.6101	1020	0.8594
1.4023	1.6417	1040	0.8591
0.8901	1.6732	1060	0.8574
1.2387	1.7048	1080	0.8575
0.9921	1.7364	1100	0.8564
1.0593	1.7680	1120	0.8558
1.3434	1.7995	1140	0.8558
0.8345	1.8311	1160	0.8554
1.3537	1.8627	1180	0.8554
1.0417	1.8942	1200	0.8552
1.0643	1.9258	1220	0.8551
1.2218	1.9574	1240	0.8551
1.1633	1.9890	1260	0.8551

Framework versions

PEFT 0.10.0
Transformers 4.40.0
Pytorch 2.2.1+cu121
Datasets 2.19.0
Tokenizers 0.19.1

Library: peft
License: apache-2.0
Created: Apr 28, 2024
Updated: Apr 28, 2024