lora-alpaca_alpagasus_ar

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This repo contains a low-rank adapter for AceGPT-7B fit on the arbml/alpagasus_cleaned_ar.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

arbml/alpagasus_cleaned_ar

Training Hyperparameters

python finetune.py --base_model 'FreedomIntelligence/AceGPT-7B' --data_path 'alpagasus_cleaned_ar.json' --output_dir 'lora-alpaca_alpagasus'
Training Alpaca-LoRA model with params:
base_model: FreedomIntelligence/AceGPT-7B
data_path: alpagasus_cleaned_ar.json
output_dir: lora-alpaca_alpagasus
batch_size: 128
micro_batch_size: 4
num_epochs: 3
learning_rate: 0.0003
cutoff_len: 256
val_set_size: 2000
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules: ['q_proj', 'v_proj']
train_on_inputs: True
add_eos_token: False
group_by_length: False
wandb_project:
wandb_run_name:
wandb_watch:
wandb_log_model:
resume_from_checkpoint: False
prompt template: alpaca

Framework versions

• PEFT 0.7.2.dev0