GeoSynth-Canny

Model Card for Model ID

This is a ControlNet based model that synthesizes satellite images given Canny edge images. The base stable diffusion model used is stable-diffusion-2-1-base (v2-1_512-ema-pruned.ckpt).

• Use it with 🧨 diffusers
• Use it with controlnet repository

<!-- Provide a quick summary of what the model is/does. -->

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: stable-diffusion
• Paper: Adding Conditional Control to Text-to-Image Diffusion Models

Examples

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
import torch
from PIL import Image

img = Image.open("canny_example.jpeg")

controlnet = ControlNetModel.from_pretrained("MVRL/GeoSynth-Canny")

scheduler = UniPCMultistepScheduler.from_pretrained("stabilityai/stable-diffusion-2-1-base", subfolder="scheduler")

pipe = StableDiffusionControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", controlnet=controlnet, scheduler=scheduler)
pipe.enable_xformers_memory_efficient_attention()
pipe.enable_model_cpu_offload()

# generate image
generator = torch.manual_seed(10345340)
image = pipe(
    "Satellite image features a city neighborhood",
    generator=generator,
    image=img,
).images[0]

image.save("generated_city.jpg")

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]