Downloads · 30 days
5.7K
24% of all-time downloads
KBlueLeaf/TIPO-100M
TIPO-100M is a text generation model from KBlueLeaf. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This 100M model is still under development
Downloads · 30 days
5.7K
24% of all-time downloads
All-time downloads
23.4K
Public
Parameters
100M
1.2 GB on disk
Likes
9
Public
Click a slice to open those files.
.safetensors402 MB · 67%
From the Hugging Face model README
This 100M model is still under development
100M LLaMA arch model trained for TIPO.<br> Tech Report: https://arxiv.org/abs/2411.08127

In this project, we introduce "TIPO" (Text to Image with text presampling for Prompt Optimization), an innovative framework designed to significantly enhance the quality and usability of Text-to-Image (T2I) generative models. TIPO utilizes the Large Language Models (LLMs) to perform "Text Presampling" within the inference pipeline of text-to-image generative modeling. By refining and extending user input prompts, TIPO enables generative models to produce superior results with minimal user effort, making T2I systems more accessible and effective for a wider range of users.
Use updated version of DTG extension (renamed to z-tipo-extension), current version of z-tipo-extension support stable-diffusion-webui, stable-diffusion-webui-forge and ComfyUI. SD-Next haven't been tested. https://github.com/KohakuBlueleaf/z-tipo-extension
This model is LLaMA arch with 200M parameters, the training data is combined version of Danbooru2023, Coyo-HD-11M. <br> The total token seen is around 50B tokens. <br> For more information please refer to the tech report and following table.
| TIPO-200M | TIPO-200M-ft | TIPO-500M | |
|---|---|---|---|
| Arch | LLaMA | LLaMA | LLaMA |
| Max ctx length | 1024 | 1024 | 1024 |
| Batch Size | 2048 | 2048 | 3584 |
| Training dataset | Danbooru, GBC10M, 5epoch<br />Danbooru, GBC10M, Coyo11M, 3epoch | Danbooru(pixtral), Coyo11M, 2epoch | Danbooru, GBC10M, Coyo11M, 5epoch |
| Real Token Seen* | 40B token | 50B (10B more from TIPO-200M) | 30B token |
| Training Hardware | RTX 3090 x 4 | RTX 3090 x 4 | H100 x 8 |
| Training Time | 420 hour` | 120 hour` | 100 hour` |
| Huggingface | KBlueLeaf/TIPO-200M · Hugging Face | KBlueLeaf/TIPO-200M-ft · Hugging Face | KBlueLeaf/TIPO-500M · Hugging Face |
*: We only count "non-padding token" in the token seen, since all the training data have very large length range. <br> `: Since the training data is pretty short, it cost more time to reach same token seen than general LLM pretraining. <br> As reference, with 4096 as max ctx length and almost all the data have reach that length, you may only need 2days to reach 10B token seen on RTX 3090 x 4 with 200M model.
Evaluation are done on TIPO-200M model <br> We have tested TIPO compared to other Model in several test and metrics:
In this test we use single "scenery" tag as input. (With some certain meta) <br> To test each prompt gen method to see if they can obtain the desired distribution of outputs while maintain the quality of images.
| Scenery Tag Test | Original | GPT4o-mini | Prompt DB | Promptis | TIPO(ours) |
|---|---|---|---|---|---|
| FDD ↓ | 0.3558 | 0.5414 | 0.3247 | 0.2350 | 0.2282 |
| Aesthetic ↑ | 5.0569 | 6.3676 | 6.1609 | 5.9468 | 6.2571 |
| AI Corrupt ↑ | 0.4257 | 0.7490 | 0.5024 | 0.5669 | 0.9195 |
In this test we use short caption or manually truncated caption from GBC10M and CoyoHD11M. <br> This test examine the ability of prompt gen method on handling almostly completed prompts.
| Short | Original | GPT4o-mini | Prompt DB | Promptis | TIPO(ours) |
|---|---|---|---|---|---|
| FDD ↓ | 0.0957 | 0.1668 | 0.0980 | 0.1783 | 0.1168 |
| Aesthetic ↑ | 5.8370 | 6.0589 | 5.8213 | 5.7963 | 5.8531 |
| AI Corrupt ↑ | 0.7113 | 0.6985 | 0.7064 | 0.6314 | 0.7131 |
| Truncated Long | Original | GPT4o-mini | Prompt DB | Promptis | TIPO(ours) |
|---|---|---|---|---|---|
| FDD ↓ | 0.0955 | 0.1683 | 0.1247 | 0.2096 | 0.1210 |
| Aesthetic ↑ | 5.7497 | 6.0168 | 5.8191 | 5.7759 | 5.8364 |
| AI Corrupt ↑ | 0.6868 | 0.6712 | 0.6741 | 0.5925 | 0.7130 |
For research purpose, this model is released under Apache-2.0 License.
@misc{yeh2024tipotextimagetext,
title={TIPO: Text to Image with Text Presampling for Prompt Optimization},
author={Shih-Ying Yeh and Sang-Hyun Park and Giyeong Oh and Min Song and Youngjae Yu},
year={2024},
eprint={2411.08127},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2411.08127},
}