gemma-prompt

gemma-prompt

This model is a fine-tuned version of google/gemma-2b on 2 datasets- dolly-15k for general knowledge and a curated m-a-p/MusicPile

Model description

This model is a completed trained model used for music knowledge and prompt automation from musical vibes.

Intended uses & limitations

Intended use for the model is to have it generate prompts for music that takes into account elements of the surrounding environment, such as the types of buildings nearby, the weather, time of day, and nearby landmarks.

Training and evaluation data

The datasets used to help train the model are the dolly-15k dataset for general purpose of answering questions and following commands, and a second curated m-a-p/MusicPile data used to fine-tune the model specifically for musical vibe and descriptions of different objects, places, and things. We evaluate our model using a portion of the m-a-p/MusicPile.

Training procedure

Split dataset from MusicPile to focus on distilled music knowledge Used dolly for general finetuning

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• training_steps: 888

Framework versions

• PEFT 0.10.0
• Transformers 4.40.0
• Pytorch 2.0.1a0+cxx11.abi
• Datasets 2.19.0
• Tokenizers 0.19.1