shape-guided-controlnet

<div align="center">

Re-implementation of ControlNet with Shape Masks

[GitHub] / [Dataset]

</div>

A re-implementation of ControlNet with shape masks.

Model Weights Structure

shape-guided-controlnet/
└── annotators                        <----- Model weights of the shape mask annotator (`U2-Net`)
    └── u2net.pth
└── shape-guided-controlnet           <----- Model weights of the trained ControlNet with shape masks
    ├── config.json
    └── diffusion_pytorch_model.safetensors
└── stable-diffusion-v1.5             <----- Model weights of Stable Diffusion v1.5
    ├── feature_extractor
    ├── scheduler
    ├── text_encoder
    ├── tokenizer
    ├── unet
    ├── vae
    ├── model_index.json
    ├── v1-5-pruned.safetensors
    └── v1-inference.yaml

Results

Here are some example results generated by the trained model:

1. "A red bag"

<div align="center"> <img src="assets/bag_total.png" alt="Bag" width="500" /> </div>

1. "A sport car"

<div align="center"> <img src="assets/sport_car_total.png" alt="Sport Car" width="500" /> </div>

1. "A blue truck"

<div align="center"> <img src="assets/truck_total.png" alt="Truck" width="500" /> </div>