Downloads · 30 days
19
4% of all-time downloads
keras-dreambooth/dreambooth_riffusion_model_currulao_v1
dreambooth_riffusion_model_currulao_v1 is a machine learning model from keras-dreambooth. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for tf-keras. The card lists the license as openrail.
Riffusion-Currulao (WIP) is a fine-tuned version of the Riffusion model, which is based on the Stable Diffusion 1.5 framework. This model is specifically designed to generate spectrograms for Currulao music, a traditi…
Downloads · 30 days
19
4% of all-time downloads
All-time downloads
438
Public
Repo size
3.5 GB
Likes
0
Public
Click a slice to open those files.
.data-00000-of-000013.4 GB · 99%
From the Hugging Face model README
Riffusion-Currulao (WIP) is a fine-tuned version of the Riffusion model, which is based on the Stable Diffusion 1.5 framework. This model is specifically designed to generate spectrograms for Currulao music, a traditional Afro-Colombian genre characterized by the use of the Marimba de Chonta, a unique wooden percussion instrument.
Currulao is an essential part of the Afro-Colombian cultural heritage and represents the rich history, traditions, and artistic expressions of the Afro-Colombian community. By leveraging the power of the Riffusion model and the additional fine-tuning through Keras using the (DreamBooth)[https://dreambooth.github.io/] technique, Riffusion-Currulao is capable of generating spectrograms that capture the essence of Currulao music.
The model was further enhanced using DreamBooth, which allowed the integration of the concepts of Currulao and Marimba de Chonta, ensuring that the generated spectrograms somehow represent the musical characteristics of this genre.
Riffusion-Currulao can be used for various applications, including music synthesis, audio analysis, and research on Afro-Colombian music and culture. It can also serve as a creative tool for artists and musicians who wish to explore Currulao music and incorporate its elements into their compositions.
The model is designed to be accessible and user-friendly, making it suitable for both professionals and enthusiasts interested in generating spectrograms of Currulao music for their projects or studies.
Prompt: A $currulao song, lo-fi, nostalgic.
Steps: 100
Guidance: 15
<img src="https://huggingface.co/juancopi81/dreambooth_riffusion_model_currulao_v1/resolve/main/A_currulao%20song_lo-fi_nostalgic_100_15.png">You need to load the Riffusion weights to use this model. Example for Colab:
!git lfs install
!git clone https://huggingface.co/juancopi81/dreambooth_riffusion_model_currulao_v1
!pip install git+https://github.com/costiash/stable-diffusion-tensorflow --upgrade --quiet
!pip install tensorflow tensorflow_addons ftfy --upgrade --quiet
!pip install pytorch-lightning
!pip install pydub
!pip install huggingface-hub
!pip install -q -U keras_cv
!pip install -q -U tensorflow
import tensorflow as tf
from keras_cv.models.stable_diffusion.clip_tokenizer import SimpleTokenizer
from keras_cv.models.stable_diffusion.diffusion_model import DiffusionModel
from keras_cv.models.stable_diffusion.stable_diffusion import StableDiffusion as StableDiffusionKeras
from keras_cv.models.stable_diffusion.image_encoder import ImageEncoder
from keras_cv.models.stable_diffusion.noise_scheduler import NoiseScheduler
from keras_cv.models.stable_diffusion.text_encoder import TextEncoder
import keras
from stable_diffusion_tf.stable_diffusion import StableDiffusion as StableDiffusionPy
resolution=512
sd_dreambooth_model_1=StableDiffusionPy(resolution, resolution, download_weights=False, jit_compile=True)
diffusion_model_pytorch_weights = keras.utils.get_file(
origin="https://huggingface.co/riffusion/riffusion-model-v1/resolve/main/riffusion-model-v1.ckpt",
file_hash="99a6eb51c18e16a6121180f3daa69344e571618b195533f67ae94be4eb135a57",
)
sd_dreambooth_model_1.load_weights_from_pytorch_ckpt(diffusion_model_pytorch_weights)
sd_dreambooth_model_1.diffusion_model.load_weights("/content/dreambooth_riffusion_model_currulao_v1")
img = sd_dreambooth_model_1.generate(
"A $currulao song, EDM",
num_steps=100,
unconditional_guidance_scale=12,
temperature=1.2,
batch_size=1,
)
pil_img = Image.fromarray(img[0])
Riffusion-Currulao (work in progress) will be intended for:
Always ensure that the use of generated spectrograms aligns with the intended purpose and respects the cultural significance of the Currulao music and Afro-Colombian heritage.
This model was fine-tuned using the DreamBooth technique. The spectrograms for fine-tuning the model were generated from the song: "MUSICA PARA RELAJAR DEL PACÍFICO". The spectrograms were generated using the "Riffusion Manipulation Tools" repo.
