Llama3-70b-Instruct-extractor-adaptor

Utshav/Llama3-70b-Instruct-extractor-adaptor

Llama3-70b-Instruct-extractor-adaptor is a machine learning model from Utshav. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

9% of all-time downloads

All-time downloads

Public

Repo size

414 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors414 MB · 98%

At a glance

Library: peft
License: other
Access: Public
Created: Apr 23, 2024
Updated: Apr 23, 2024
SHA: 71ab30d5

Base models

meta-llama/Meta-Llama-3-70B-Instruct

Datasets

generator

Model card

From the Hugging Face model README

Llama2-70b-Instruct-finetuned

This model is a fine-tuned version of meta-llama/Meta-Llama-3-70B-Instruct on the generator dataset. It achieves the following results on the evaluation set:

Loss: 0.6518

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2.5e-05
train_batch_size: 1
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 0.03
training_steps: 500

Training results

Training Loss	Epoch	Step	Validation Loss
1.4651	0.0305	10	1.4614
1.2857	0.0610	20	1.3825
1.3614	0.0915	30	1.3106
1.1613	0.1220	40	1.2354
1.1483	0.1524	50	1.1536
1.1123	0.1829	60	1.0586
1.0873	0.2134	70	0.9631
0.8711	0.2439	80	0.8951
0.8709	0.2744	90	0.8485
0.8406	0.3049	100	0.8152
0.8181	0.3354	110	0.7901
0.8111	0.3659	120	0.7741
0.7284	0.3963	130	0.7599
0.7409	0.4268	140	0.7474
0.6901	0.4573	150	0.7374
0.6622	0.4878	160	0.7287
0.6912	0.5183	170	0.7212
0.7425	0.5488	180	0.7156
0.7135	0.5793	190	0.7095
0.7233	0.6098	200	0.7040
0.7314	0.6402	210	0.6985
0.659	0.6707	220	0.6916
0.6884	0.7012	230	0.6870
0.6891	0.7317	240	0.6845
0.6736	0.7622	250	0.6813
0.6487	0.7927	260	0.6784
0.5908	0.8232	270	0.6755
0.6864	0.8537	280	0.6728
0.6581	0.8841	290	0.6688
0.6816	0.9146	300	0.6667
0.6503	0.9451	310	0.6648
0.6625	0.9756	320	0.6626
0.6392	1.0061	330	0.6616
0.6319	1.0366	340	0.6613
0.6228	1.0671	350	0.6613
0.5918	1.0976	360	0.6606
0.6028	1.1280	370	0.6589
0.6563	1.1585	380	0.6569
0.6154	1.1890	390	0.6556
0.5797	1.2195	400	0.6545
0.6137	1.25	410	0.6538
0.6174	1.2805	420	0.6533
0.5981	1.3110	430	0.6528
0.5793	1.3415	440	0.6526
0.5626	1.3720	450	0.6523
0.5864	1.4024	460	0.6520
0.5874	1.4329	470	0.6519
0.6221	1.4634	480	0.6518
0.5727	1.4939	490	0.6517
0.5776	1.5244	500	0.6518

Framework versions

PEFT 0.10.1.dev0
Transformers 4.41.0.dev0
Pytorch 2.1.1+cu121
Datasets 2.19.0
Tokenizers 0.19.1

Library: peft
License: other
Created: Apr 23, 2024
Updated: Apr 23, 2024