pom_klementieff_lora_flux

Pom Klementieff Lora Flux

<Gallery />

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

This a non-quantized version of https://huggingface.co/je-suis-tm/pom_klementieff_lora_flux_nf4. Both are trained on the same dataset. The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux.py. The training took 3 hours on A100 80GB with max VRAM consumption at 35GB. The inference consumes 36GB of VRAM.

Train

    export MODEL_NAME="black-forest-labs/FLUX.1-dev"
    export INSTANCE_DIR="/pvol/pom_klementieff"
    export OUTPUT_DIR="/pvol/pom_klementieff_lora_flux"
    accelerate config default
    accelerate launch train_dreambooth_lora_flux1.py \
      --pretrained_model_name_or_path=$MODEL_NAME  \
      --mixed_precision="bf16" \
      --dataset_name=$INSTANCE_DIR \
      --output_dir=$OUTPUT_DIR \
      --gradient_checkpointing \
      --instance_prompt="Pom Klementieff" \
      --caption_column="text" \
      --resolution=1024 \
      --train_batch_size=1 \
      --guidance_scale=1 \
      --use_8bit_adam \
      --checkpointing_steps=100 \
      --gradient_accumulation_steps=4 \
      --optimizer="adamW" \
      --learning_rate=1e-4 \
      --lr_scheduler="constant" \
      --lr_warmup_steps=100 \
      --max_train_steps=1500 \
      --rank=4 \
      --seed="0" 

Usage

import torch
from diffusers import FluxPipeline

device = "cuda:0"

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)

pipe.load_lora_weights("je-suis-tm/pom_klementieff_lora_flux",
                       weight_name='pytorch_lora_weights.safetensors')


prompt = "female parachutist named Pom Klementieff, jumping from plane, smile, selfie, motion, full body visible, detailed skin, realistic, photo-realistic, 8k, highly detailed, full length frame, High detail RAW color art, diffused soft lighting, shallow depth of field, sharp focus, hyperrealism, cinematic lighting"

image = pipe(
  prompt=prompt,
  generator=torch.Generator(device=device).manual_seed(42),
  num_inference_steps=50, # 28 is a good trade-off
  guidance_scale=4,
  height=1024, 
  width=1024,
).images[0]

image.save("pom_klementieff.png")

Trigger words

You should use Pom Klementieff to trigger the image generation.

Download model

Download them in the Files & versions tab.