neosumi

NeoSumi

The text encoder can predict the logits behind the mask tokens.

Regardless of the captioning strategy, it can understand both booru captions and natural language.

The loss was calculated in the ForMaskedLM mode during training.

Anime images

The encoded text can be used with compatible, uncensored image generation models.

Masking syntax

<doc>word1 <mask>word2 ...

The tokens are often "word + whitespace". In the example, do not put a space between the mask token and the next word.

Setup

pip install "git+https://github.com/huggingface/transformers.git@4b24a197a1fa996496e3cf4f476c140e083218b5"

The model requires either the dev branch or future version v5.17 of the transformers library.

Base model

The NeoMME with 500k steps was chosen as the base model, not the predecay variant with 450k steps.

Tokenization

The tokenizer has remained unchanged.

The English stop words have been removed from the captions.

The comma-seperated keywords were scrambled and adjusted for the model.

Initial epochs

The model was initialised using a low learning rate and natural language captions from CC (Moondream) and Danbooru (Qwen3.5).

The masked language model then continued to learn from a combination of comma-separated and natural language texts.

The artists' given names, nicknames and fantasy names have been deliberately excluded from the dataset, as it would be difficult to predict these based on the text alone.

Fixed danbooru

Grio's patched tags were applied to the danbooru dataset. This makes it the most recent and accurate source.

Other booru sites were included as well.

Source data

• anime_captions
• artbench_captions
• cc12m_2mp_realistic
• danbooru_multitier_captions_202606
• dtg_character_tags
• furry_e621
• gelbooru_characters_enriched
• gelbooru_tags_full
• tag_cleaning
• wikiart_215k
• samples for a certain locked tag