EasyFluffXL

PSA: Avoid using Beta scheduler for the Vpred checkpoints on Forge/ReForge if it produces broken outputs. May be hardware specific and not effect everyone. Possibly related to a very recent commit.

Prompting Tip: While the model can be prompted with tags, natural language, and a mix of the two, the merging process has resulted in overlap of tags that mean one thing and the same words that can mean something else. If you find a certain character tag weak or otherwise "off", place a comma at the end of it. (Example: "ikari shinji," instead of just "ikari shinji") A quirk of fine-tuned tag-based models is that the commas that seperate the tags in the training captions matter when prompting the resulting model. Commas aren't always necessary, especially if you're able to give supporting words that act as reinforcment for a specific tag, but sometimes a comma is needed for a full character likness. This can extend to artist tags and more niche concept tags.

[November 18th, 2024]

Upon further inspection, the original V-Prediction 0.5 version of NoobAI-XL officially added natural language captions to their dataset and I've observed that the raw model now responds to natural language prompts better than 1.0. Due to this, I'm holding off on any further merging and letting the upstream V-Prediction model become more trained as that will offer much more versatility for future merging with the goal of better prompt adherance with natural language prompting.

<strike>[November 15th, 2024]

EasyFluffXL-V added, a re-merged V-Prediction checkpoint that should allow higher CFG without burning and collapsing. 4-6 CFG range seems ideal now instead of 3-4. Ancestral samlpers are still quite sensitive to higher CFG, so be careful. I advise avoiding VpredIter and Vpred checkpoints but they will be left up for posterity.</strike>

[November 14th, 2024]

Uploaded Redo version of Pre. Acts similar but should have less artifacting.

[November 12th, 2024]

Pending.

[November 11th, 2024]

EasyFluffXLVpred/Iter is a more experimental checkpoint that utliizes NoobAI-XL's 0.5 V-Prediction checkpoint to convert the model to Vpred, which brings some advantages ranging from better colors and better prompt following. It also merges in Itercomp for the Iter version. Epsilon-prediction checkpoints are a WIP as it's hard to bring parity in performance between the two. LoRas trained on Noob 1.0 should still work adequately on this merge. If training a LoRA, I still reccomend training directly on NoobAI-XL 1.0.

Generally safe settings are ~4 CFG (higher than 4 can become noticably too saturated without rescale), Euler A, and Normal scheduler.

I reccomend using an up-to-date ComfyUI. As of https://github.com/comfyanonymous/ComfyUI/commit/8b275ce5be29ff7d847c3c4c2f3fea1faa68e07b ComfyUI will also automatically detect that the model is V-Prediction/ZSNR and set it for you. If for some reason the autodetection doesn't work or you are on an older ComfyUI, use the ModelSamplingDiscrete node to set V-Prediction and ZSNR. Alternatively your option is an up-to-date Forge/ReForge as they automatically detect the vpred and tznsr key I've added to the state_dict of both models and inference as V-Prediction/ZSNR. Forge currently doesn't have a RescaleCFG extension built in, but ReForge does. However I have included a backport of RescaleCFG for Forge in this repo that may work unless the code has changed since creating it. Extract it into extensions-builtin. Another option is A1111's dev branch, as it also has auto-detection of V-Prediction SDXL checkpoints from my understanding.

[Old]

Experimental merge of NoobAI-XL 1.0, introducing natural language prompting. No Lightning or other low-step bake-ins. Too early to tell, but I do notice it performs measurably worse at multi-subject outputs that exceed more than 2-3 characters. Some tag associations get lost in meaning with natural language being present now. Despite that, I find the current state of the merge "fun". As there is most likely room for improvement, what I will initially upload will be with the caveat that it's an early version and in no way final. Euler A + Beta scheduler seems reasonably safe. Due to the merging process and addition of natural language prompting, some lower tag-count characters and concepts may require some upweighting and/or supporting tags. Some NSFW interactions require really delicate prompting to get what you want.

This model is give or take capable of the same content as NoobAI-XL 1.0, which consists of most if not all of Danbooru and E621 posts, given the respective cutoff dates. Meaning this checkpoint can and will produce NSFW content!