Downloads · 30 days
0
phi-2-alpaca-lora
xezno/phi-2-alpaca-lora
phi-2-alpaca-lora is a machine learning model from xezno. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
This is a low-rank adapter for phi-2 fit on the alpaca dataset.
Access
Public
Updated Dec 17, 2023
Repo size
189 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt126 MB · 65%
At a glance
- License
- other
- Access
- Public
- Created
- Dec 17, 2023
- Updated
- Dec 17, 2023
- SHA
- e1cae7ab
Datasets
Model card
From the Hugging Face model README
Model Card for phi-2-alpaca
This is a low-rank adapter for phi-2 fit on the alpaca dataset.
Training Hyperparameters
The model was trained on 1xA100 GPU using PEFT-LORA.
The following hyperparameters were used during training:
- Lora target modules: Wqkv, out_proj
- Lora r: 16
- lora_alpha: 16
- lora_dropout: 0.1
- learning_rate: 5e-05
- per_device_train_batch_size: 1
- gradient_accumulation_steps: 1
- training_steps: 120000
Limitations and Bias
The model is based on a large and diverse dataset, but it may still have limitations and biases in certain areas. Some limitations include:
- Language: The model is designed to work with English text only and may not perform as well in other languages.
In addition, the model may have some bias in terms of the data it was trained on. The dataset includes questions from a variety of sources, but it may not be representative of all populations or perspectives. As a result, the model may perform better or worse for certain types of questions or on certain types of texts.
- License
- other
- Created
- Dec 17, 2023
- Updated
- Dec 17, 2023