krea2-character-lora-recipe

Krea 2 Character LoRA — Training Recipe

End-to-end recipe for training a character (or style) LoRA on Krea-2-Raw using musubi-tuner (kohya-ss), verified on a single-session full run on 96GB VRAM (RTX PRO 6000, Blackwell sm_120).

Above: one preview generated with this recipe's LoRA on Krea-2-Turbo. The character is "Queen Jedi (QJ)" — a private VRChat avatar of mine. See the Showcase gallery below for more.

---

What this is — and what it isn't

This is a training recipe, not a model release. What you get:

• The full training pipeline, parameters, and scripts used in a real session.
• Configuration templates (configs/) you adapt to your own data.
• Lessons learned from the run, with measured numbers.

What is NOT here:

• ❌ The trained LoRA file (.safetensors) — kept private.
• ❌ The training dataset — kept private (photos of my own VRChat character "Queen Jedi").
• ❌ A way to download anything you can plug into ComfyUI directly.

The preview images in samples/ (if present) are a showcase of what this exact recipe produces with proper data. They aren't reproducible without your own dataset, but they tell you whether the recipe is worth your time.

---

Who this is for

• You have ~96GB VRAM available: RTX PRO 6000 / H100 80GB+swap / H200 / A100 80GB (with caveats) / DGX Spark.
• You have a coherent dataset (200-500+ images, 1:1 .txt captions).
• You want a character LoRA that holds identity through scenes and poses on Krea-2-Raw.

For lower VRAM (24-48GB), see the Lower VRAM section — you trade speed for memory via --gradient_checkpointing and --fp8_base --fp8_scaled.

---

TL;DR

# 1) Drop your dataset (images + 1:1 .txt captions) into a folder, e.g.:
#    ~/musubi-tuner/datasets/MY_CHAR/

# 2) Adapt configs (paths, batch_size, trigger token):
nano configs/dataset_example.toml
nano configs/sample_prompts_example.txt

# 3) Pre-cache latents + text encoder outputs (once):
bash scripts/precache_krea2.sh

# 4) Full run (16 epochs):
bash scripts/run_krea2_train.sh
tail -f ~/musubi-tuner/output/MY_RUN/run_*.log

# 5) Plot loss (any time, raw + 100-step moving average):
python scripts/plot_loss.py

# 6) Resume from last state if interrupted:
bash scripts/run_krea2_train.sh resume

---

Hardware requirements

Stop any GPU process before launching — the trainer wants the entire VRAM budget.

---

Models you need

You need four files. The trainer is musubi-tuner (clone separately).

Use hf download (the new HuggingFace CLI), not curl — multipart-parallel gets ~35 MB/s, plain curl often throttles below 5 MB/s on large HF files:

HF_HUB_ENABLE_HF_TRANSFER=1 \
  hf download krea/Krea-2-Turbo turbo.safetensors \
  --local-dir ~/musubi-tuner/models/krea2/

For gated krea/Krea-2-Raw, set HF_TOKEN env first.

---

Dataset and captions

Drop your images + matching .txt caption files (1:1, same basename) into your dataset folder. Example: cat.png + cat.txt.

Caption rule that matters: describe only what is visible in the frame.

• Close-up portrait → don't mention the full outfit, shoes, tail.
• Full-body shot → describe the full outfit including trigger tokens.
• Same logic for environment, props, accessories.

This trains the model to bind trigger tokens to visible attributes, not just to the character's full description. It's the difference between a LoRA that holds together across compositions and one that smears the trigger across the canvas.

Regularization images (optional but recommended for character LoRAs): a second [[datasets]] block in the TOML, mixing in 1:1 with the main set, helps prevent overfitting on subject features.

See configs/dataset_example.toml for the full template.

---

Sample prompts (training-time previews)

configs/sample_prompts_example.txt — one prompt per line, with CLI overrides at the end. The launcher uses --sample_every_n_epochs 1 --sample_at_first, so you get a 5-image preview grid every epoch.

Turbo-style preview overrides (matches --turbo_dit flag in the launcher):

--w 1024 --h 1024 --s 8 --l 1 --d 42

• --s 8 — 8 sampling steps (Turbo)
• --l 1 — CFG off (Turbo requires this)
• --d N — fixed seed per prompt (stable identity across epochs)
• --mu 1.15 — implicit from --turbo_dit, no need to pass

If you run previews through Raw (without --turbo_dit): use --s 28 --l 5.5 --d N. Raw at CFG=1 produces intentionally blurry images by design — this is not a bug.

---

Pre-cache (run once)

Two stages baked into scripts/precache_krea2.sh:

1. Latents — VAE encodes every image into cache/<name>_<res>_kr2.safetensors.
2. TE outputs — Qwen3-VL encodes every caption into <name>_kr2_te.safetensors.

After this, the trainer doesn't load VAE/TE at each step — the cache is on disk, training is pure DiT forward/backward. Typical throughput: ~15 it/s for the precache pass, ~1 minute for 800 image-caption pairs.

Re-run only if the dataset or captions changed.

---

Training launch

scripts/run_krea2_train.sh [mode]. Modes:

The launcher embeds these tuned parameters (measured on real runs):

---

Monitoring

# Live log
tail -f ~/musubi-tuner/output/MY_RUN/run_*.log

# GPU
watch -n 1 nvidia-smi

# Loss plot (raw + 100-step moving average → PNG):
python scripts/plot_loss.py

# Or interactive TensorBoard:
tensorboard --logdir ~/musubi-tuner/logs --port 6006
# open http://localhost:6006

Preview filenames: <name>_e<epoch>_<promptIdx>_<timestamp>_<seed>_<batchIdx>.png. Epoch tells you when; seed tells you which prompt (42-46 for the default 5).

---

Resume

State is written to output/MY_RUN/MY_RUN-e<N>-state/ at every save checkpoint. The launcher's resume mode picks the latest state automatically:

bash scripts/run_krea2_train.sh resume

To resume from a specific state (not the latest):

accelerate launch \
  <musubi-tuner>/src/musubi_tuner/krea2_train_network.py \
  <all the usual flags from the launcher> \
  --resume <musubi-tuner>/output/MY_RUN/MY_RUN-e000005-state

Optimizer, scheduler, and LoRA weights all restore exactly — you continue from the same step with the same internal state.

---

Memory — measured, not theoretical

On RTX PRO 6000 96GB VRAM, dry-run measurements:

Krea-2 is a big DiT (features=6144, 28 wide blocks, multiplier=4). At bs=2 without gradient checkpointing, forward+backward activations alone are ~60GB.

Choosing your config:

• Speed > gradient smoothness → bs=1, gc=off (default; ~4.5h for 16 epochs)
• Smoothest gradients → bs=4 + gc on (theoretically ~5.5h)
• OOM anywhere → enable --gradient_checkpointing (−30% speed, −50GB activations)

---

Bucket aspect-ratio impact (practical, not in official docs)

A side observation from running this recipe on two different datasets:

A ~15-17% wall-time penalty comes from bucket re-allocation when training batches alternate between shapes. The activations get re-laid-out, the cuDNN kernels re-pick algos, and you pay for it on every transition.

Actionable: if you want maximum throughput, normalize your dataset to a single aspect ratio (crop / pad / square-up) before training. The trainer won't complain about mixed aspects, but you do pay for the variety.

Not a deal-breaker for character LoRAs — variety in poses and crops is more valuable than 17% off the clock. But worth knowing.

---

Real-world measurements (full 16-epoch run, 2026-06-26)

A real run (not dry-run), 474-image main set + 348 regularization, bs=1, gc=off:

Dry-run showed 55-65GB; the full run sat at 72GB. The difference is most likely --turbo_dit_cache keeping mappings/buffers around in VRAM even when the Turbo weights themselves are parked in CPU RAM. The 24GB headroom held through all 16 epochs and all sample cycles — no OOM.

---

Smaller-dataset datapoint

A second run on the same hardware with a ~6.5× smaller dataset, to give readers a rough scaling sense for their own work:

Same config across both runs: bs=1, gradient_checkpointing off, AdamW 1e-4, sdpa, network_dim 32 / alpha 32, 16 epochs, save_every_n_epochs 1, sample_every_n_epochs 1.

The higher final loss on Run B is expected — fewer total steps means training stops sooner, before the loss curve has fully flattened. Quality is still solid in practice (identity holds, character stacks cleanly with style LoRAs) — final loss is a useful indicator but not the only one. If your dataset is in the 100-200 image range, plan for roughly 1 hour of wall time on 96GB and expect a slightly higher loss floor than the larger-dataset run; both can produce strong LoRAs.

---

Troubleshooting

---

Lower VRAM (24-48GB)

This recipe is tuned for 96GB. For smaller GPUs, the path is:

• 48GB (RTX 6000 Ada, A6000): --gradient_checkpointing + bs=1 fits with margin.
• 24GB (RTX 4090/5090): --gradient_checkpointing + bs=1 + --fp8_base --fp8_scaled (halves DiT to ~12.5GB). Expect quality variance vs. fp16/bf16 base.
• <24GB: offload more (CPU offloading via Accelerate config) or accept unreasonable wall times. Krea-2 is just a big model.

I haven't measured these myself — if you run them, a follow-up issue / PR with your measurements is welcome.

---

Repository layout

.
├── README.md                            # this file
├── LICENSE                              # CC BY 4.0
├── .gitattributes                       # Git LFS rules (images / safetensors)
├── scripts/
│   ├── precache_krea2.sh                # one-time latent + TE cache builder
│   ├── run_krea2_train.sh               # main launcher (run / resume / dryrun / fg)
│   └── plot_loss.py                     # TB events → raw + MA-100 PNG
├── configs/
│   ├── dataset_example.toml             # neutral dataset config template
│   └── sample_prompts_example.txt       # neutral preview prompts template
└── samples/                             # showcase preview images (CC BY 4.0)

Scripts contain hardcoded paths from the original setup (/home/jahjedi/musubi-tuner/...). Adjust them to your environment before running.

---

About the showcase

The preview images in samples/ come from training this recipe on a private dataset of my own VRChat character "Queen Jedi (QJ)" — a demon-queen persona with purple skin, blonde hair, curved horns, a floating crown, and a tail. I own both the avatar and the photographs used as training data.

Neither the dataset nor the trained LoRA are part of this repository. The previews are here only to show what this recipe can produce; reproducing them requires your own data.

Each image was generated with the trained QJ LoRA stacked on top of Krea-2-Turbo, often combined with a community style LoRA at strength 0.6-0.8. The QJ LoRA held identity consistently at 0.8 weight even under heavy style LoRA mixing — a sanity check that the recipe is not over-fitting.

Gallery

---

Sources

• musubi-tuner: https://github.com/kohya-ss/musubi-tuner
• Krea-2-Raw model card: https://huggingface.co/krea/Krea-2-Raw
• Krea-2 doc in musubi: https://github.com/kohya-ss/musubi-tuner/blob/main/docs/krea2.md
• Comfy-Org single-files: https://huggingface.co/Comfy-Org/Qwen3-VL, https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI

---

License

Recipe, scripts, configs, README, and preview images: CC BY 4.0 — see LICENSE.

The trained LoRA file and training dataset referenced in this recipe are not part of this repository and are not licensed for distribution.

---

_{Workflow notes and write-up assisted by Claude.}