Downloads · 30 days
0
alpaca-lora-30b
chansung/alpaca-lora-30b
alpaca-lora-30b is a text generation model from chansung. Use it when you need the model to write or continue text. The card lists the license as gpl-3.0.
This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX sys…
Access
Public
Updated Apr 6, 2023
Repo size
460 MB
Likes
49
Public
Repo makeup
Click a slice to open those files.
.bin205 MB · 100%
At a glance
- Task
- Text Generation
- License
- gpl-3.0
- Access
- Public
- Created
- Mar 19, 2023
- Updated
- Apr 6, 2023
- SHA
- 597694d6
Try a prompt
Datasets
Model card
From the Hugging Face model README
This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system.
- Dataset: cleaned-up Alpaca dataset up to 04/06/23
- Training script: borrowed from the official Alpaca-LoRA implementation
- Training script:
python finetune.py \
--base_model='decapoda-research/llama-30b-hf' \
--num_epochs=10 \
--cutoff_len=512 \
--group_by_length \
--output_dir='./lora-alpaca' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
--lora_r=16 \
--batch_size=... \
--micro_batch_size=...
- Task
- Text Generation
- License
- gpl-3.0
- Languages
- en
- Created
- Mar 19, 2023
- Updated
- Apr 6, 2023