Downloads · 30 days
0
alpaca-lora-65b
acctouhou/alpaca-lora-65b
alpaca-lora-65b is a machine learning model from acctouhou. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is just a test based on the lora 65b model. Used for the MIT NLP class final project. Then there will be three steps: - Calculate and accumulate gradients - Determine the appropriate rank through gradient computa…
Access
Public
Updated Dec 4, 2023
Repo size
126 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin169 MB · 50%
At a glance
- License
- mit
- Access
- Public
- Created
- Dec 4, 2023
- Updated
- Dec 4, 2023
- SHA
- 58148241
Model card
From the Hugging Face model README
This is just a test based on the lora 65b model. Used for the MIT NLP class final project. Then there will be three steps:
- Calculate and accumulate gradients
- Determine the appropriate rank through gradient computation
- Perform LORA fine-tuning.
LORA fine-tuning
For 24G VRAM on GPT2_SM model (Original version of Lora)
python main.py --train_batch_size 8 --valid_batch_size 8 --grad_acc 1 --model_card gpt2.SM --init_checkpoint pretrained_checkpoints/gpt2-pytorch_model.bin --work_dir alpha_sm --index 0
For 24G VRAM on GPT2_SM model (Our version of Lora)
python main.py --train_batch_size 8 --valid_batch_size 8 --grad_acc 1 --model_card gpt2.SM --init_checkpoint pretrained_checkpoints/gpt2-pytorch_model.bin --work_dir alpha_sm --index 1
license: mit
- License
- mit
- Created
- Dec 4, 2023
- Updated
- Dec 4, 2023