erasedraw-v0.2

These are the model weights for EraseDraw.

<img src="https://erasedraw.cs.columbia.edu/static/img/samples_overview.png" alt="Sample Overview">

To use this model, install diffusers using main for now. The API is the same as that of InstructPix2Pix

pip install diffusers accelerate safetensors transformers

import PIL
import requests
import torch
from diffusers import StableDiffusionInstructPix2PixPipeline, EulerAncestralDiscreteScheduler
model_id = "alpercanberk/erasedraw"
pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(model_id, torch_dtype=torch.float16, safety_checker=None)
pipe.to("cuda")
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
url = "https://raw.githubusercontent.com/timothybrooks/instruct-pix2pix/main/imgs/example.jpg"
def download_image(url):
    image = PIL.Image.open(requests.get(url, stream=True).raw)
    image = PIL.ImageOps.exif_transpose(image)
    image = image.convert("RGB")
    return image
image = download_image(url)
prompt = "add sunglasses"
images = pipe(prompt, image=image, num_inference_steps=10, image_guidance_scale=1).images
images[0]

Code and data are coming soon to GitHub (find link on website).

You may also try out the other model they have slight differences.