ASUKA-FLUX.1-Fill

<p align="center"> <h1 align="center">Towards Enhanced Image Inpainting:<br> Mitigating Unwanted Object Insertion and Preserving Color Consistency</h1> <center>Yikai Wang*, Chenjie Cao*, Junqiu Yu*, Ke Fan, Xiangyang Xue, Yanwei Fu†.<br> Fudan University<br> <b>CVPR 2025 <font color="#ed7748">(Highlight)</font></b> </center> <p align="center"> <a href="https://arxiv.org/abs/2312.04831"><img alt='arXiv' src="https://img.shields.io/badge/arXiv-2312.04831-b31b1b.svg"></a> <a href="https://yikai-wang.github.io/asuka/"><img alt='page' src="https://img.shields.io/badge/Project-Website-orange"></a> </p> <br> </p>

Overview

This repo contains the proposed ASUKA model in our paper "Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency".

ASUKA solves two issues existed in current diffusion and rectified flow inpainting models: <b>Unwanted object insertion</b>, where randomly elements that are not aligned with the unmasked region are generated; <b>Color-inconsistenc</b>y, the color shift of the generated masked region, causing smear-like traces. ASUKA proposes a post-training procedure for these models, significantly mitigates object hallucination and improves color consistency of inpainted results.

We released ASUKA for FLUX.1-Fill-dev, denoted as ASUKA(FLUX.1-Fill). The code and dataset can be found at here. We are actively working to improve both our model and evaluation dataset. If you encounter failure cases with ASUKA (FLUX.1-Fill) or have challenging examples in image inpainting, we would love to hear from you. Please email them to [email protected]. We truly appreciate your contributions!

Modifications to FLUX

• The text conditional input of CLIP and T5 is replaced by the MAE condition to mitigate object hallucination.
• The decoder is replaced by our conditional decoder to enhance color consistency.

BibTeX

If you find our repo helpful, please consider cite our paper :)

@inproceedings{wang2025towards,
  title={Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency.},
  author={Wang, Yikai and Cao, Chenjie and Yu, Junqiu and Fan, Ke and Xue, Xiangyang and Fu, Yanwei},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
  year={2025}
}