scepter_scedit

<p align="center"><h2 align="center">🪄SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing</h2> </p> <p align="center"> <strong>Zeyinzi Jiang</strong> · <strong>Chaojie Mao</strong> · <strong>Yulin Pan</strong> · <strong>Zhen Han</strong> · <strong>Jingfeng Zhang</strong> </p> <div align="center">

Paper (ArXiv) | Project Page | Code| Swift

</div> <p align="center"> <b>Alibaba Group</b> </p> <p> <table align="center"> <tr> <td> <img src="assets/figures/show.jpg"> </td> </tr> </table> </p>

SCEdit is an efficient generative fine-tuning framework proposed by Alibaba TongYi Vision Intelligence Lab. This framework enhances the fine-tuning capabilities for text-to-image generation downstream tasks and enables quick adaptation to specific generative scenarios, saving 30%-50% of training memory costs compared to LoRA. Furthermore, it can be directly extended to controllable image generation tasks, requiring only 7.9% of the parameters that ControlNet needs for conditional generation and saving 30% of memory usage. It supports various conditional generation tasks including edge maps, depth maps, segmentation maps, poses, color maps, and image completion.

Use Models

pip install scepter
python -m scepter.tools.webui

BibTeX

@article{jiang2023scedit,
    title = {SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing},
    author = {Jiang, Zeyinzi and Mao, Chaojie and Pan, Yulin and Han, Zhen and Zhang, Jingfeng},
    year = {2023},
    journal = {arXiv preprint arXiv:2312.11392}  
}