Downloads · 30 days
10
40% of all-time downloads
llm_finetune
alikehuggie/llm_finetune
llm_finetune is a text generation model from alikehuggie. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
All-time downloads
25
Public
Repo size
17.1 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin17.1 KB · 86%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Dec 1, 2023
- Updated
- Dec 1, 2023
- SHA
- 02dc7900
Try a prompt
Datasets
Model card
From the Hugging Face model README
Base model
bigscience/bloomz-560m
Training procedure
According to edX Databricks llm102 course
PromptTuningConfig
- task_type=TaskType.CAUSAL_LM,
- prompt_tuning_init=PromptTuningInit.RANDOM,
- num_virtual_tokens=4,
TrainingArguments
- learning_rate= 3e-2, # Higher learning rate than full fine-tuning
- num_train_epochs=5 # Number of passes to go through the entire fine-tuning dataset
Framework versions
- PEFT 0.4.0
Training output
TrainOutput(global_step=35, training_loss=3.386413792201451, metrics={'train_runtime': 617.1546, 'train_samples_per_second': 0.405, 'train_steps_per_second': 0.057, 'total_flos': 58327152033792.0, 'train_loss': 3.386413792201451, 'epoch': 5.0})
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Languages
- en
- Created
- Dec 1, 2023
- Updated
- Dec 1, 2023