science-sobotta-anatomical-drawings

DreamBooth model for for anatomical drawings

This is a Stable Diffusion model fine-tuned on the sbtstyle concept with DreamBooth. It can be used by modifying the instance_prompt: an anatomical drawing in the style of sbtstyle

This model was created as part of the DreamBooth Hackathon 🔥. Visit the organisation page for instructions on how to take part!

Examples and data

<table> <tr> <td>One of the images used to fine-tune on<br>"an anatomical drawing in the style of sbtstyle"</td> <td>One of the images generated by the model<br>" an anatomical drawing of a bicycle in the style of sbtstyle"</td> <td>One of the images generated by the model<br>" an anatomical drawing of a horse in the style of sbtstyle"</td> </tr> <tr> <td> <img src="https://i.imgur.com/nALP2kQ.png" style="max-height:300px"> </td> <td> <img src="https://i.imgur.com/JGg2Zvs.png" style="max-height:300px"> </td> <td> <img src="https://i.imgur.com/cNpfXjI.png" style="max-height:300px"> </td> </tr> </tr></table>

Some more examples generated by others:

<table> <tr> <td> <img src="https://i.imgur.com/fOht9A9.jpg" style="max-height:300px"> </td> <td> <img src="https://i.imgur.com/GHCRf6n.png" style="max-height:300px"> </td> <td> </td> </tr> </table>

Dataset

The full dataset used to train on can be found here: Sanderbaduk/sobotta-anatomical-dataset. These images and others like it can be found on wikimedia commons.

If you use this directly you will get an error as the raw images have different numbers of channels. Include a grayscale step to fix this and ensure that classic feel.

self.transforms = transforms.Compose(
    [
        transforms.Resize(size),
        transforms.Grayscale(num_output_channels=3), 
        transforms.CenterCrop(size),
        transforms.ToTensor(),
        transforms.Normalize([0.5], [0.5]),
    ]
)

Usage

from diffusers import StableDiffusionPipeline

pipeline = StableDiffusionPipeline.from_pretrained('Sanderbaduk/science-sobotta-anatomical-drawings')
image = pipeline().images[0]
image