sd3-dreambooth-lora-cube

SD3 DreamBooth LoRA - AdamLucek/sd3-cube-dreambooth-lora-cube

Reference Image

<img src="https://cdn-uploads.huggingface.co/production/uploads/65ba68a15d2ef0a4b2c892b4/Fx9Py6PeZL6Jaj4yKk8dl.jpeg" width=400>

Generated Images

<img src="https://cdn-uploads.huggingface.co/production/uploads/65ba68a15d2ef0a4b2c892b4/UYl02nsBCddvQSB5-egxC.jpeg" width=400>

Model description

These are DreamBooth LoRA weights for stabilityai/stable-diffusion-3-medium-diffusers.

Trigger words

You should use A photo of T0K cube to trigger the image generation.

Download model

Download the pytorch_lora_weights.safetensors LoRA in the Files & versions tab.

Use it with the 🧨 diffusers library

from diffusers import AutoPipelineForText2Image
import torch
pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3-medium-diffusers', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('AdamLucek/sd3-cube-dreambooth-lora-2', weight_name='pytorch_lora_weights.safetensors')
image = pipeline('A photo of T0K cube on a shelf').images[0]

Use it with UIs such as AUTOMATIC1111, Comfy UI, SD.Next, Invoke

• LoRA: download diffusers_lora_weights.safetensors here 💾.
  • Rename it and place it on your models/Lora folder.
  • On AUTOMATIC1111, load the LoRA by adding <lora:your_new_name:1> to your prompt. On ComfyUI just load it as a regular LoRA.

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers

License

Please adhere to the licensing terms as described here.

Training details

The weights were trained using DreamBooth with the SD3 diffusers trainer.

Script

accelerate launch train_dreambooth_lora_sd3.py \
  --pretrained_model_name_or_path="stabilityai/stable-diffusion-3-medium-diffusers"  \
  --output_dir="lora-trained-sd3-t3" \
  --dataset_name="AdamLucek/cube-pics-dreambooth" \
  --mixed_precision="fp16" \
  --instance_prompt="A photo of T0K cube" \
  --resolution=1024 \
  --train_batch_size=16 \
  --learning_rate=1e-4 \
  --report_to="wandb" \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=1000 \
  --validation_prompt="A photo of T0K cube on a shelf" \
  --validation_epochs=500 \
  --seed="420" \
  --rank=16 \
  --hub_model_id="sd3-cube-dreambooth-lora-2" \
  --train_text_encoder \
  --push_to_hub

Trained on a single H100