Downloads · 30 days
0
RWKV-Piano-Experiments
smashmaster/RWKV-Piano-Experiments
RWKV-Piano-Experiments is a machine learning model from smashmaster. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gpl-3.0.
Experiments on training 0.4B RWKV models around midi notation in a manner similar to this already existing midi model.
Access
Public
Updated Apr 12, 2024
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth2.5 GB · 100%
At a glance
- License
- gpl-3.0
- Access
- Public
- Created
- Mar 23, 2024
- Updated
- Apr 12, 2024
- SHA
- feaf2301
Model card
From the Hugging Face model README
Experiments on training 0.4B RWKV models around midi notation in a manner similar to this already existing midi model.
- RWKV v4neo based, 20 epoch: Loss of about 2.7ish
- WIP v6 pretrain that also sucks. Loss was around 2.3 to 2.5 but I'm guessing it ended up at 2.5, kind of sad but this can be used as a base I guess?
April 12, 2024 Update
- Added v6 with different layer sizes.
- Trained a base model on all of bread midi filtered by piano instrument only augumented 10 times. See the following wandb for training logs (note experimentation, finalish runs are used for the final file).
- Used above model as the initial model and then trained on a combined dataset of Breadmidi + Los Angeles + Monster filtered by piano augumented 3x (wish I could have the storage space to do more). See the following wandb.
- License
- gpl-3.0
- Created
- Mar 23, 2024
- Updated
- Apr 12, 2024