HonyararaMix

ご利用の際は下記のライセンス内容を十分にご確認ください。

HonyararaMixはアニメ調の人物を出力しやすいように調整されたStable Diffusion 2.1 解像度768対応版モデルです。
ベースモデルのIlluminati Diffusion 1.0へWaifu Diffusion 1.5 Beta 、Waifu Diffusion 1.5 Beta2、Replicant-V1.0、Stable Diffusion 2.1を用いマージしたものをベースに
DeDeDeDatasetによる学習などを行い調整したものです。

旧HonyararaMixにはライセンス上の問題があったため、それらを解決したモデルになります。

Prompt/Bad Promptについては、ベースモデルとなった二種のモデルが用意した文書の内容、およびTI Embeddingがこのモデルでも作用します。
WDについてはこちらを参照。Illuminati DiffusionについてはNfixerが割とよく効く印象です。

例

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image01.png" style="max-width:1000px;" width="75%"/>

best quality, masterpiece, great nature and secret garden with anime 1girl with twintail green hair and green eye and beautiful iris and (((from behind)))
Negative prompt: nfixer,3d, flat shading, flat shadow, bad anatomy, bad hands, missing fingers, extra digit, fewer digits looking at viewer
Steps: 28, Sampler: DDIM, CFG scale: 9, Seed: 43662843, Size: 1024x768, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image02.png" style="max-width:1000px;" width="75%"/>

newest, masterpiece, exceptional, anime game cg of cute cute cute 1girl sitting on a bench in harajuku park with detailed wavy pink hair pink and detailed yellow hair
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 28, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 4179369813, Size: 768x1152, Model hash: 4dbf24eef6, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image03.png" style="max-width:1000px;" width="75%"/>

newest, masterpiece, exceptional, anime game cg of kawaii samurai 1girl, fighting form and (((intricate hands))), fist and dynamic angle with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
Negative prompt: nfixer, 3d, flat shading, flat shadow, ((bad hands, missing fingers, extra digit, fewer digits))
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7.5, Seed: 2246223688, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image04.png" style="max-width:1000px;" width="75%"/>

((masterpiece, best quality)),best aesthetic, 1girl, solo, black skirt, blue eyes, guitar, headphones, instrument, long hair, music, one side up, pink hair, playing guiter, pleated skirt, black shirt, indoors
Negative prompt: nfixer,missing finger, extra digits, fewer digits,((mutated hands and fingers)), 3d, flat shadow
Steps: 16, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3689716537, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image05.png" style="max-width:1000px;" width="75%"/>

newest, masterpiece, exceptional, anime game cg of teenage 1boy wizard bust shot casting fire magic spell in New York City, detailed eye and beautiful iris,picturesque, golden hour, dynamic pose
Negative prompt: nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 25, Sampler: Euler a, CFG scale: 7.5, Seed: 1414427875, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

<img src="https://huggingface.co/nakayama/HonyararaMix/resolve/main/img/image06.png" style="max-width:1000px;" width="75%"/>

newest, masterpiece, exceptional, anime game cg of samurai (((((old man))))), fighting pose, dynamic angle, fist with detailed pink hair and detailed yellow eye and beautiful iris in japanese background
nfixer, 3d, flat shading, flat shadow, inaccurate limb, bad hands, missing fingers, extra digit, fewer digits
Steps: 25, Sampler: DDIM, CFG scale: 9, Seed: 3970412380, Size: 768x1152, Denoising strength: 0.35, Clip skip: 2, ENSD: 31337, Hires upscale: 2, Hires upscaler: R-ESRGAN 4x+ Anime6B

マージ・学習手順について

1. Illuminati Diffusion 1.0にWaifu Diffusion 1.5 Beta1を差分マージする
   | Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name | | --- | --- | --- | --- | --- | | Add Difference @ 0.6 | Illuminati Diffusion 1.0 | Waifu Diffusion 1.5 Beta1 | Stable Diffusion v2-1_768-ema-pruned | HonyararaMix_pre1 |

2. HonyararaMix_pre1に対し、自前で用意した他Diffusion Modelの出力からなる素材画像にて学習させる
   用意の際に利用したサービス/モデルは、SD2.1、Novel AI、WD1.3/1.4、CoolJapan Diffusion 2.1、Dreamlike Photoreal 2.0。
   使用したソフトはkohya版Fine Tuning。
   総数は30000程、flipしたものと合わせてlearning rateは5e-6、以下の設定で学習させ、
   試行錯誤の後にepoch 2相当のものをHonyararaMix_pre2とした。

accelerate launch --num_cpu_threads_per_process 1 fine_tune.py
  --pretrained_model_name_or_path=Honyarara_pre1.safetensors
  --in_json meta_lat.json --train_data_dir=train_data
  --output_dir=fine_tuned
  --shuffle_caption
  --train_batch_size=4
  --learning_rate=5e-6
  --max_train_steps=60000
  --v2
  --v_parameterization
  --use_8bit_adam
  --xformers
  --mixed_precision=bf16
  --save_every_n_epochs=2
  --save_precision=float
  --max_token_length=150
  --seed=42

1. Honyarara_pre1にHonyarara_pre2を加重平均でマージする
   | Interpolation Method | Primary Model | Secondary Model | Merge Name | | --- | --- | --- | --- | | Weighted Sum @ 0.5 | HonyararaMix_pre1 | HonyararaMix_pre2 | HonyararaMix_pre3 |

2. HonyararaMix_pre3にIlluminati Diffusion 1.0を階層マージする
   使用マージ用スクリプトはSuper Merger。
   | Model: A | Model: B | Weight | Merge Name | | --- | --- | --- | --- | | HonyararaMix_pre3 | Illuminati Diffusion 1.0 | 1.0,0.45,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.1,0.3,0.5,0.7,0.9,1.0 | HonyararaMix_pre4|

3. HonyararaMix_pre4にWaifu Diffusion 1.5 Beta2 Aestheticを差分マージする
   | Interpolation Method | Primary Model | Secondary Model | Tertiary Model | Merge Name | | --- | --- | --- | --- | --- | | Add Difference @ 0.5 | HonyararaMix_pre4 | WD 1.5 Beta 2 Aesthetic | WD 1.5 Beta 2 | HonyararaMix_pre5 |

4. HonyararaMix_pre5にReplicant-V1.0を加重平均でマージする
   | Interpolation Method | Primary Model | Secondary Model | Merge Name | | --- | --- | --- | --- | | Weighted Sum @ 0.5 | HonyararaMix_pre5 | Replicant-V1.0 | HonyararaMix A |

5. HonyararaMix AにHonyararaMix_pre4を加重平均でマージする
   | Interpolation Method | Primary Model | Secondary Model | Merge Name | | --- | --- | --- | --- | | Weighted Sum @ 0.5 | HonyararaMix A | HonyararaMix_pre4 | HonyararaMix B |

ライセンスについて

Fair AI Public License 1.0-SD (https://freedevproject.org/faipl-1.0-sd/)