Downloads · 30 days
0
alpaca-lora-7b
spelt/alpaca-lora-7b
alpaca-lora-7b is a machine learning model from spelt. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This repo contains a low-rank adapter for LLaMA-7b fit on the Stanford Alpaca dataset.
Access
Public
Updated Apr 10, 2023
Repo size
151 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin67.2 MB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Apr 10, 2023
- Updated
- Apr 10, 2023
- SHA
- 8b1620c9
Datasets
Model card
From the Hugging Face model README
This repo contains a low-rank adapter for LLaMA-7b fit on the Stanford Alpaca dataset.
This version of the weights was trained with the following hyperparameters:
- Epochs: 10 (load from best epoch)
- Batch size: 128
- Cutoff length: 512
- Learning rate: 3e-4
- Lora r: 16
- Lora target modules: q_proj, k_proj, v_proj, o_proj
That is:
python finetune.py \
--base_model='decapoda-research/llama-7b-hf' \
--num_epochs=10 \
--cutoff_len=512 \
--group_by_length \
--output_dir='./lora-alpaca-512-qkvo' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
--lora_r=16 \
--micro_batch_size=8
Instructions for running it can be found at https://github.com/tloen/alpaca-lora.
- License
- mit
- Created
- Apr 10, 2023
- Updated
- Apr 10, 2023