dptlab-sdxl-lora-toy-v1

dptlab-sdxl-lora-toy-v1

Post-trained with dptlab using the lora recipe on top of stabilityai/stable-diffusion-xl-base-1.0.

Training config

{
  "model_key": "sdxl",
  "recipe": "lora",
  "dataset_path": "data/concept_dataset",
  "output_dir": "/root/outputs/lora-sdxl",
  "resolution": 1024,
  "learning_rate": 0.0001,
  "train_batch_size": 1,
  "gradient_accumulation_steps": 4,
  "max_train_steps": 20,
  "lora_rank": 16,
  "lora_alpha": 16,
  "mixed_precision": "bf16",
  "seed": 42,
  "checkpointing_steps": 20,
  "validation_prompts": [
    "a photo of sks concept in a forest",
    "sks concept, studio lighting, product photo"
  ],
  "validation_steps": 250,
  "extra": {
    "resume_from": "/root/outputs/lora-sdxl/final"
  }
}

Benchmark

See MODELS.md in the repo for the full leaderboard across checkpoints.

Usage

from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights("OnePunchMonk101010/dptlab-sdxl-lora-toy-v1")

image = pipe(prompt="your prompt here").images[0]