falcon-7b-alpaca

amztheory/falcon-7b-alpaca

falcon-7b-alpaca is a machine learning model from amztheory. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

3% of all-time downloads

All-time downloads

305

Public

Repo size

37.8 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors37.8 MB · 93%

At a glance

Library: peft
License: apache-2.0
Access: Public
Created: May 24, 2024
Updated: May 25, 2024
SHA: 862dcf0a

Base models

tiiuae/falcon-7b-instruct

Model card

From the Hugging Face model README

falcon-7b-alpaca

This model is a fine-tuned version of tiiuae/falcon-7b-instruct on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.9657

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 6
eval_batch_size: 6
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 24
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
1.3865	0.0631	50	1.4032
1.2768	0.1262	100	1.2262
1.021	0.1893	150	1.0728
1.0843	0.2524	200	1.0175
0.9927	0.3155	250	0.9966
0.9746	0.3785	300	0.9836
1.02	0.4416	350	0.9767
0.9806	0.5047	400	0.9722
1.0579	0.5678	450	0.9684
1.0075	0.6309	500	0.9670
0.9909	0.6940	550	0.9661
0.9792	0.7571	600	0.9661
0.9449	0.8202	650	0.9654
0.9268	0.8833	700	0.9656
1.0441	0.9464	750	0.9657

Framework versions

PEFT 0.11.1
Transformers 4.41.0
Pytorch 2.3.0+cu121
Datasets 2.19.1
Tokenizers 0.19.1

Library: peft
License: apache-2.0
Created: May 24, 2024
Updated: May 25, 2024