stable-signature-bzh-sdxl-vae-strong

Watermarked (strong) VAE for SDXL

You can use this watermarked VAE for your existing SDXL image generation pipelines to make sure generated images are watermarked.

Usage

from diffusers.models import AutoencoderKL
from diffusers import StableDiffusionXLPipeline

model = "stabilityai/sdxl-turbo"

vae = AutoencoderKL.from_pretrained("imatag/stable-signature-bzh-sdxl-vae-strong")

pipe = StableDiffusionXLPipeline.from_pretrained(model, vae=vae)

For more information, please have a look at the official demo

Purpose

This model was finetuned from the original StableDiffusion-XL autoencoder with the additional objective of including a strong invisible watermark, following the procedure of StableSignature. This watermark is detectable by IMATAG's demo BZH decoder, available via API.

Compared to other watermarking solutions such as invisible-watermark:

• the watermark is quite robust to many unintentional editing transforms such as cropping, resizing, changing contrast or brightness, and JPEG compression
• the watermark has zero overhead in terms of computational power
• the watermark is embedded in the model weights and not as a post-processing of the non-watermarked image
• the probability of detecting the watermark by chance on non-watermarked images may be computed analytically and limited to a specific threshold

Detecting the watermark

For security reasons we cannot release the weights of the detector. Instead, you may check for the presence of a watermark with the detect_api.py script like this:

python detect_api.py test.png

Visual

Visualization of watermark impact on 512x512 images from the COCO2017 validation dataset prompts with SDXL-turbo.

<p align="center"> <br> <b> 512x512: original (left), watermarked (middle), difference (right)</b> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0001.png /> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0003.png /> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0004.png /> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0005.png /> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0006.png /> </p>

Robustness

The watermark is robust to most editorial changes to an image, such as cropping, changing brightness or contrast, resizing or JPEG compression:

<p align="center"> <br> <b> from left to right: original, watermarked, difference, 'crop 50% + brigthen 50% + jpeg 80%' attack, 'downscale 2x + crop 50% + brigthen 50% + jpeg 50%' attack</b> </p> <p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0000.png /> </p>

The Receiver Operating Characteristic curve shows its performance under no modification and when modified by the two scenarios illustrated above:

<p align="center"> <img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/roc.png /> </p>

More details available in our announcement and our lab's blog post. For watermarked models with a different key, support for payload, other perceptual compromises, robustness to other attacks, or faster detection, please contact IMATAG.