animagine-xl-4.0-opt-pvc-style-lora

PVC Style LoRA for AnimagineXL 4.0 opt

<Gallery />

Model description

Only tested with ComfyUI.

Trigger words

You should use pvc to trigger the image generation.

You should use nendoroid to trigger the image generation.

You should use figma to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Dataset

About 7000 of pvc figure images.

Training

Training code

https://github.com/p1atdev/vision-ft

Training settings

<details> <summary>`config.yml`</summary>

model:
  checkpoint_path: "./models/animagine-xl-4.0-opt.bnb_nf4.safetensors"
  pretrained_model_name_or_path: cagliostrolab/animagine-xl-4.0

  dtype: bfloat16

  denoiser:
    attn_implementation: "flash_attention_2"

peft:
  config:
    type: lora
    rank: 2
    alpha: 1.0
    dropout: 0.0

    dtype: bfloat16

  include_keys:
    - "attn1"
    - "attn2"
    - ".ff."
  exclude_keys: ["text_encoder", "vae"]

dataset:
  folder: "data/pvc"
  num_repeats: 4
  batch_size: 4

  bucket_base_size: 1024
  step: 128
  min_size: 384
  do_upscale: true

  caption_processors:
    - type: shuffle
      split_separator: ","

optimizer:
  name: "schedulefree.RAdamScheduleFree"
  args:
    lr: 0.005

scheduler:
  # name: "torch.optim.lr_scheduler.ConstantLR"
  # args: {}

tracker:
  project_name: "sdxl-pvc-1"
  loggers:
    - wandb

saving:
  strategy:
    per_epochs: 0.25
    per_steps: null
    save_last: true

  callbacks:
    - type: "safentensors"
      name: "sdxl-pvc"
      save_dir: "./output/sdxl-pvc-2"

preview:
  strategy:
    per_epochs: 1
    per_steps: 100

  callbacks:
    - type: "local"
      save_dir: "./output/sdxl-lora/preview"

  data:
    path: "./projects/pvc-sdxl/preview.yml"

seed: 42
num_train_epochs: 10

trainer:
  # debug_mode: "1step"

  gradient_checkpointing: true

  torch_compile: true
  torch_compile_args:
    backend: eager
    mode: default
    fullgraph: false

  fp32_matmul_precision: "high"
  allow_tf32: true

</details> <details> <summary>`preview.yml` </summary>

- prompt: |-
    pvc style, 1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, 
    green background, hat, hoop earrings, jewelry, looking at viewer, 
    shirt, short hair, simple background, solo, upper body, yellow shirt, 
    masterpiece, high score, great score, absurdres
  negative_prompt: |-
    lowres, bad anatomy, bad hands, text, error, missing finger, extra digits, 
    fewer digits, cropped, worst quality, low quality, low score, bad score, 
    average score, signature, watermark, username, blurry
  height: 1024
  width: 1024
  cfg_scale: 5.0
  num_steps: 20

  seed: 0

</details>

The training was stopped in the 9th epoch, because of my room was too hot due to this training.

Device and Hours

RTX 4070 Ti Super (VRAM 16GB) for 27 hours.