Downloads · 30 days
3
17% of all-time downloads
bloom_LoRA_tuning_1696763384.8620448
vinclab/bloom_LoRA_tuning_1696763384.8620448
bloom_LoRA_tuning_1696763384.8620448 is a machine learning model from vinclab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
All-time downloads
18
Public
Repo size
410 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin410 KB · 99%
At a glance
- Library
- peft
- Access
- Public
- Created
- Oct 8, 2023
- Updated
- Oct 8, 2023
- SHA
- 67ab579c
Model card
From the Hugging Face model README
Model
"bigscience/bloomz-560m"
Dataset
"Abirate/english_quotes" (English quotes)
Training procedure
PEFT using LoRA
lora configuration: - r=1, - lora_alpha=3, - target_modules=["query_key_value"], - lora_dropout=0.05, - bias="none", - task_type="CAUSAL_LM"
trainable params: 98,304 || all params: 559,312,896 || trainable%: 0.01757585078102687 None
training with Pytorch using the folowing parameters: - learning_rate= 1e-2 - num_train_epochs=10
Framework versions
- PEFT 0.4.0
- Library
- peft
- Created
- Oct 8, 2023
- Updated
- Oct 8, 2023