click2mask

[AAAI 2025] Click2Mask: Local Editing with Dynamic Mask Generation

Official Model Card for "Click2Mask: Local Editing with Dynamic Mask Generation".

Paper by: Omer Regev, Omri Avrahami, Dani Lischinski

<br><br>

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/teaser.gif" alt="Click2Mask Teaser"/>

Given an image, a <b>Click</b>, and a prompt for an added object, a Mask is generated dynamically, simultaneously with the object generation throughout the diffusion process.

Current methods rely on existing objects/segments, or user effort (masks/detailed text), to localize object additions. Our approach enables free-form editing, where the manipulated area is not well-defined, using just a <b>Click</b> for localization.

🚀 Try Click2Mask Online

🤗 Hugging Face Demo

Try it instantly in your browser - no setup required.

⚡ Google Colab Demo

Includes both Gradio interface and command line for advanced usage.

For additional usage options, please refer to our repository.

Examples

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/results.jpg" alt="Click2Mask Results" width="900"/>

Qualitative Comparisons with SoTA Methods

A brief glimpse into the qualitative comparison of SoTA methods — Emu Edit, MagicBrush and InstructPix2Pix — against our model, Click2Mask. <br> Upper prompts were given to baselines, and lower (shorter) ones to Click2Mask. Inputs contain the <b>Click</b> given to Click2Mask.

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/compare.png" alt="Comparison" width="900"/>

Evaluating Edited Regions in Maskless Methods

We introduce Edited Alpha-CLIP to evaluate mask-free methods by extracting a mask of the edited region and using Alpha-CLIP to assess its alignment with the prompt. <br> Examples of mask extractions: outputs are on the left, extracted masks (green overlay) on the right.

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/edited_alphaclip.png" alt="Comparison" width="900"/>

Citation

If you find this helpful for your research, please reference the following:

@inproceedings{regev2025click2mask,
    title={Click2Mask: Local Editing with Dynamic Mask Generation},
    author={Regev, Omer and Avrahami, Omri and Lischinski, Dani},
    booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
    volume={39},
    number={7},
    pages={6713-6721},
    year={2025},
    url={https://arxiv.org/abs/2409.08272},
    note={Full version with appendices available on arXiv}
}

Acknowledgements

Our code is based on Blended Latent Diffusion and Stable Diffusion, and utilizes AlphaCLIP (this model card contains a mirror of AlphaCLIP's weights).