Mugen-Jina-V1.5

Mugen with Jina-clip-v2 + adapter as text encoder

• Previous version
• UNet trained significantly more

Note:

• Current version is still WIP. I'm not 100% finished with this current version. There's definitely things I need to fix / limitations with this version. But, I'm uploading it for now, just to get something out.

Details for Usage:

• ComfyUI Custom nodes, make sure to update to the latest.
  • If using sage-attention, make sure to switch to sage-attention-triton (you can use KJ nodes to do this). As the default sage-attention doesn't support the cross-attention mask.
• Forge Neo Extension
• Rest is same as before
• PDG custom node included at extension/block_drop_guidance.py, used to increase sharpness of outputs. (Just place in extensions folder of ComfyUI)
  • Current final finetune was trained with slight PDG dropout (0.05).

Adapter changes:

• Cross-attention and adapter masking of padding tokens.
  • (Padding is now used to pad to nearest 77 chunk as sage-attention relies on this)
• Removed the additional positional embeddings from the adapter. This was a mistake on my part to add to the original. The position of the text now has less of an impact on the output generation.
• Works with, and trained with prompts up to 1024 tokens.
• More training

Credits:

• DeepGHS: Used danbooru2024-webp-4MPixel
• Cabal Research: Creators of Mugen
• Minthy: T5-gemma adapter + Anime-Art-Multicaptions-v5.0 NLP dataset.
• NeuroSenko: Used their ComfyUI custom node code as a base.