Evo-Nishikie-v1

🐟 Evo-Nishikie-v1

🤗 Models | 📝 Blog | 🐦 Twitter

Evo-Nishikie-v1 is an experimental education-purpose Ukiyoe colorization model. The model is a ControlNet trained with Evo-Ukiyoe. The dataset used to train the model came from Ukiyoe images belonged to Ritsumeikan University, Art Research Center. The sample data is belonged to the Pre-Modern Japanese Text dataset, owned by National Institute of Japanese Literature and curated by ROIS-DS Center for Open Data in the Humanities.

Please refer to our blog for more details.

Usage

Use the code below to get started with the model.

<details> <summary> Click to expand </summary>

1. Git clone this model card

   git clone https://huggingface.co/SakanaAI/Evo-Nishikie-v1
   

2. Install git-lfs if you don't have it yet.

   sudo apt install git-lfs
   git lfs install
   

3. Create conda env

   conda create -n evo-nishikie python=3.11
   conda activate evo-nishikie
   

4. Install packages

   cd Evo-Nishikie-v1
   pip install -r requirements.txt
   

5. Run

   import torch
   from io import BytesIO
   from PIL import Image
   import requests
   from evo_nishikie_v1 import load_evo_nishikie
   
   #Get image from URL
   #url = "https://huggingface.co/spaces/SakanaAI/Evo-Nishikie/resolve/main/sample2.jpg"
   #original_image = Image.open(BytesIO(requests.get(url).content))
   
   #Use local image
   original_image = Image.open('test.jpg')
   
   # Generate
   device = "cuda"
   pipe, processor = load_evo_nishikie(device)
   images = pipe(
        prompt="着物を着た女性が、赤ん坊を抱え、もう一人の子どもが手押し車を引いています。背景には木があります。最高品質の輻の浮世絵。超詳細。",
        negative_prompt="暗い",
        image=processor(original_image),
        guidance_scale=7.0,
        controlnet_conditioning_scale=0.8,
        num_inference_steps=35,
        num_images_per_prompt=1,
        output_type="pil",
   ).images
   images[0].save("out.png")
   
   

</details>

Model Details

<!-- Provide a longer summary of what this model is. -->

• Developed by: Sakana AI
• Model type: Diffusion-based image-to-image generative model
• Language(s): Japanese
• Blog: https://sakana.ai/evo-ukiyoe

License

The Python script included in this repository is licensed under the Apache License 2.0. Please note that the license for the model/pipeline generated by this script is inherited from the source models. The sample images used in the code has CC BY SA 4.0 license and is belonged to the Pre-Modern Japanese Text dataset, owned by National Institute of Japanese Literature and curated by ROIS-DS Center for Open Data in the Humanities.

Uses

This model is provided for research and development purposes only and should be considered as an experimental prototype. It is not intended for commercial use or deployment in mission-critical environments. Use of this model is at the user's own risk, and its performance and outcomes are not guaranteed. Sakana AI shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. Users must fully understand the risks associated with the use of this model and use it at their own discretion.

Acknowledgement

Evo-Nishikie was trained based on Evo-Ukiyoe and Evo-Ukiyoe was trained based on Evo-SDXL-JP. We would like to thank the developers of Evo-SDXL-JP source models for their contributions and for making their work available.

• SDXL
• Juggernaut-XL-v9
• SDXL-DPO
• JSDXL

Citation

@misc{Evo-Ukiyoe, 
url    = {[https://huggingface.co/SakanaAI/Evo-Nishikie-v1](https://huggingface.co/SakanaAI/Evo-Nishikie-v1)}, 
title  = {Evo-Nishikie}, 
author = {Clanuwat, Tarin and Shing, Makoto and Imajuku, Yuki and Kitamoto, Asanobu and Akama, Ryo}
}