Downloads · 30 days
0
fine-tuned-flan-t5-base-v2
BhushanM25/fine-tuned-flan-t5-base-v2
fine-tuned-flan-t5-base-v2 is a machine learning model from BhushanM25. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
loraconfig = LoraConfig( r=32, Rank loraalpha=32, targetmodules=["q", "v"], loradropout=0.05, bias="none", tasktype=TaskType.SEQ2SEQLM FLAN-T5 )
Access
Public
Updated Dec 19, 2023
Repo size
14.2 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin14.2 MB · 100%
At a glance
- Access
- Public
- Created
- Dec 19, 2023
- Updated
- Dec 19, 2023
- SHA
- 9fd77d47
Model card
From the Hugging Face model README
lora_config = LoraConfig( r=32, # Rank lora_alpha=32, target_modules=["q", "v"], lora_dropout=0.05, bias="none", task_type=TaskType.SEQ_2_SEQ_LM # FLAN-T5 )
peft_training_args = TrainingArguments(
output_dir=output_dir,
auto_find_batch_size=True,
learning_rate=1e-3, # Higher learning rate than full fine-tuning.
num_train_epochs=1,
logging_steps=10,
max_steps=100
)
peft_trainer = Trainer( model=peft_model, args=peft_training_args, train_dataset=tokenized_datasets["train"], eval_dataset=tokenized_datasets['validation'] )
- Created
- Dec 19, 2023
- Updated
- Dec 19, 2023