krea2-style-lora-pair

Krea 2 Style LoRA Pair — a controlled study of low-rank style adaptation

Two rank-32 LoRA adapters for the Krea 2 flow-matching text-to-image DiT, trained as a matched pair so that their inference-time strengths compose as an approximately linear interpolation axis rather than as two unrelated knobs.

The adapters themselves are ordinary style LoRAs. The point of this repository is the experiment around them: exposure matching between adapters, a 5 × 5 strength ablation with a fixed prompt and fixed seed, and measured failure thresholds for where prompt conditioning loses to adapter priors.

中文版模型卡在本文档下半部分 — see 中文版 below.

---

1. Motivation

Stacking two style LoRAs at inference time is common practice, and the usual mental model is that strength_A = 0.5, strength_B = 0.5 yields a 50/50 blend. That model is wrong unless the two adapters saw the same amount of gradient signal during training. An adapter trained on 96 images for 10 epochs has been pushed further from the base weights than one trained on 67 images for 10 epochs, so equal strengths do not mean equal contribution.

This pair was therefore trained with matched exposure, defined as

exposure = n_images × num_repeats × epochs

The two totals differ by 7.5%, which is the residual after choosing integer repeat counts. With this in place, the strength pair (m, f) behaves as a usable continuous axis (Section 5).

2. Model details

Alpha is set equal to rank deliberately: the LoRA scaling factor becomes alpha / rank = 1, so the learning rate keeps its plain meaning and no mental conversion is needed when comparing runs.

3. Training data

Both datasets are third-party artwork collected for private research and personal use, and are not redistributed here. mretsis is game CG stills; freestyle is illustration work by online artists. No commercial licence was obtained for either corpus, no images are included in this repository, and the adapters are published for method-documentation purposes rather than as a licensed commercial product. Users are responsible for their own compliance with the base-model licence and with applicable copyright law in their jurisdiction.

Preprocessing. Aspect-ratio bucketing at a 1024² area target (enable_bucket = true, bucket_no_upscale = true) — images are never cropped and never upscaled, so composition and native detail are preserved. Latents and text embeddings are both pre-cached before training (Section 4).

Captioning policy. Captions are long natural-language descriptions, one per image, each prefixed with the trigger token. The governing rule is deliberately inverted from the usual advice:

Whatever you want to be able to change at inference time must be written in the caption. Whatever should stay constant must not be.

Anything named in the caption stays steerable by the prompt; anything left unnamed sinks into the trigger token and becomes part of what the adapter always does. For these two style adapters, subject clothing, pose, background, expression and shot size are captioned (and therefore remain promptable), while the painterly rendering characteristics are not captioned (and therefore become the adapter's fixed contribution). Example caption:

mretsis. A light-blue-haired woman in a torn black crop top ... braces one leg
forward over a body lying on the ground ... The setting is a narrow alley between
tall buildings at night. Wide shot from a low angle, her body filling the right
two thirds of the frame.

4. Training procedure

4.1 Two-stage pre-caching as a VRAM strategy

Latents (VAE) and text embeddings (Qwen3-VL-4B, ~8 GB) are cached to disk in two separate passes before training starts. This is not a speed optimisation — it is the reason the run fits in 32 GB without block swapping: neither the VAE nor the text encoder is ever resident during the training loop, so the whole budget goes to the DiT.

Consequence to be aware of: if a caption changes, the text-embedding cache must be rebuilt, or training silently continues on the stale embedding.

4.2 Hyperparameters

accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 \
  src/musubi_tuner/krea2_train_network.py \
  --dit krea2_raw_bf16.safetensors \
  --network_module networks.lora_krea2 --network_dim 32 --network_alpha 32 \
  --learning_rate 1e-4 --optimizer_type adamw8bit \
  --timestep_sampling krea2_shift \
  --sdpa --gradient_checkpointing --fp8_base --fp8_scaled \
  --max_train_epochs 10 --save_every_n_epochs 1

In-training sampling was deliberately disabled. --turbo_dit inherits --fp8_scaled and fails against an already-quantised checkpoint (Layer blocks.0.attn.gate.weight is already in torch.float8_e4m3fn format); getting previews would have required downloading a 24.5 GB bf16 Turbo model. Evaluating each saved checkpoint directly in the production ComfyUI graph is both cheaper and a more faithful measurement, since it is the actual deployment path.

4.3 Infrastructure and measured throughput

RTX 5090, 32 GB — rented on-demand (Runpod, community tier)
2.55 s/it · batch 1 · 1024² bucketed · fp8 + gradient checkpointing
VRAM 24.8 / 32.6 GB (no block swap)
mretsis   2680 steps ≈ 2 h 00 m
freestyle 2880 steps ≈ 2 h 09 m

5. Results

5.1 Loss carries no usable quality signal

<p align="center"> <img src="assets/loss_mretsis.png" width="45%" alt="mretsis training loss"/> <img src="assets/loss_freestyle.png" width="45%" alt="freestyle training loss"/> </p>

Both runs oscillate in the 0.07–0.09 band for the entire training and are, to the eye, horizontal lines. This is expected for a flow-matching / denoising objective — the loss is dominated by the sampled timestep and noise draw, not by how well the style has been acquired. No early-stopping or model-selection decision in this project was made from the loss curve. Checkpoint selection was done by generating from every saved epoch in the production graph.

5.2 Convergence is much earlier than the schedule assumed

At 67 images / rank 32 / lr 1e-4, the adapter is essentially converged by epoch 2 (268 steps). The loss flattens around steps 600–800, and epochs 2 through 10 are visually indistinguishable — verified at two different inference strengths (0.5 and 0.8). Ten epochs were budgeted; four to five would have been sufficient.

The caveat is the one that motivated the pair design: adapters intended to be stacked must still be trained to equal exposure, so shortening one run means adjusting num_repeats on the other to match.

5.3 Strength ablation (5 × 5, fixed prompt, fixed seed)

The ablation swept a 5 × 5 grid of mretsis × freestyle strengths, each in {0.4, 0.6, 0.8, 1.0, 1.2}, under one fixed prompt and two fixed seeds (4101 / 4102, same trends on both) — 50 images in total.

The figure below shows seed 4101 with the freestyle axis swept in full (0.4 → 1.2) at the two lowest mretsis weights, which includes the general-use production setting m0.6 + f1.0. The mretsis ≥ 0.8 rows are omitted from the figure, not from the analysis: as mretsis rises it progressively overrides the prompt's garment description — first opening the neckline, then the bodice — which is itself the finding reported in Sections 5.4 and 7. The rows shown isolate the freestyle end of the axis, where the background-density and ornamentation effects are clearest.

The pair forms a continuous, interpretable axis:

loose background · simpler garments        detailed background · ornamented garments
   mretsis  ●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━●  freestyle
        m1.2+f0.4    m0.8+f0.8    m0.6+f1.0    m0.4+f1.2

• mretsis-dominant: strong background defocus, simplified clothing, subject occupies more of the frame, warmer palette — the compositional conventions of its game-CG corpus.
• freestyle-dominant: denser background detail, garments grow their own ornamentation (sashes, gold trim, edge lining), tidier dress, a more illustrative finish.
• The axis is asymmetric. mretsis has the larger marginal influence: a small weight already pushes the background out of focus, whereas freestyle must reach a high weight to pull it back into focus. The left half of the axis is steeper than the right.

5.4 Where prompt control breaks

Above a combined strength of 1.6 the style portion of the prompt is effectively inert — backgrounds are uniformly bokeh and explicitly prompted hair colour and clothing can be overwritten by the adapter prior. This is the honest cost of the corpus's visual character, and it is a conditioning failure, not a rendering failure.

The practical resolution is not to lower quality expectations but to move the description into the prompt and lower the weight, so that prompt and adapter assert the same thing rather than competing. Production settings landed at m0.6 + f1.0 for general use and m0.8 + f0.8 for the higher-contrast look.

5.5 An adapter prior can silently delete a prompt token

Across 248 generations in a related ablation, the prompt token single side braid was almost never rendered: both adapters' corpora are dominated by long loose wavy hair, and that prior overrides the token. The token is not ignored by the text encoder — it loses to the adapter. Attributes that conflict with a strong dataset prior need considerably more prompt mass (multiple mutually reinforcing clauses) than attributes that agree with it.

6. Usage

ComfyUI, loading the adapters onto the DiT only:

UNETLoader(krea2 ...)
  → LoraLoaderModelOnly(mretsis_krea2_v1.safetensors,  strength 0.6)
  → LoraLoaderModelOnly(freestyle_krea2_v1.safetensors, strength 1.0)
  → KSampler(steps 12, cfg 1.0, sampler euler, scheduler simple)

Trigger tokens: mretsis. and freestyle., placed at the start of the prompt (captions were written with the token in that position). Prompts are long-form natural language, not comma-separated tags — this is what the base model and the captions were both built around.

Reference configurations: m0.6 + f1.0 (general), m0.8 + f0.8 (higher contrast). Keep the combined strength at or below 1.2.

Alternative distribution

The identical weights are also published on Civitai, whose model pages carry a larger set of community-contributed sample generations and serve as a second download route:

• freestyle_krea2_v1 · mretsis_krea2_v1

Those pages are a general-audience image-generation community and include mature content.

7. Limitations and intended use

Intended use. Research and personal creative work: reproducing the method, studying low-rank adaptation of flow-matching image models, and non-commercial image generation under the base model's licence.

Out of scope. Commercial use, imitation of any identifiable living artist's signature style as a service, and any use that would infringe the rights of the source-artwork holders. These adapters were trained on unlicensed third-party artwork (Section 3) and are not offered as a cleared commercial asset.

Known limitations.

1. Strength is not free — above a combined 1.6 the adapters override explicit prompt content (Section 5.4). mretsis in particular carries the exposure conventions of its source corpus: from about 0.8 upward it progressively overrides prompted garment coverage, and by 1.2 anatomy itself degrades. These rows are not shown in the ablation figure.
2. Dataset priors override some tokens — hairstyles and other attributes that contradict the corpus need disproportionate prompt mass (Section 5.5).
3. Corpus bias — both datasets are overwhelmingly young female figures in a stylised anime/game idiom. The adapters do not generalise to other subjects, and they carry the exposure and framing conventions of their sources.
4. Stored in fp32 — 447 MiB per file, roughly twice what is needed; casting to bf16 on save would halve it at no measurable quality cost.
5. Single base model — evaluated only on Krea 2 and on one community fine-tune of it. Behaviour on other Krea 2 derivatives is untested.
6. Qualitative evaluation only — checkpoint and strength selection were made by visual inspection under fixed seeds and single-variable ablation. No quantitative image-quality or style-similarity metric was computed, and no human study was run.

8. Reproducing

train_lora.sh in this repository is the exact script used, including the two pre-caching passes and the argument rationale as inline comments. Datasets are not included; supply /workspace/datasets/<name>/ with images and same-stem .txt captions and run:

bash train_lora.sh style mretsis   67 4    # 2680 steps, ~2 h
bash train_lora.sh style freestyle 96 3    # 2880 steps, ~2 h

Citation

@misc{krea2_style_lora_pair,
  title  = {Krea 2 Style LoRA Pair: exposure-matched low-rank style adapters
            and a strength-composition ablation},
  year   = {2026},
  note   = {LoRA adapters for the Krea 2 flow-matching text-to-image model},
  howpublished = {Hugging Face model repository}
}

---

---

中文版模型卡

Krea 2 画风 LoRA 对 —— 一次关于低秩画风适配的对照实验

两个 rank-32 LoRA，底模是 Krea 2 流匹配文生图 DiT。它们是成对训练的：曝光量被刻意找平，使得推理时两个强度旋钮构成一条近似线性的插值轴，而不是两个互不相干的开关。

适配器本身只是普通的画风 LoRA。这个仓库真正的内容是围绕它们做的实验：适配器之间的曝光量找平、固定提示词与固定种子下的 5 × 5 强度消融，以及实测得到的提示词控制权失效阈值。

1. 动机

推理时叠两个画风 LoRA 很常见，通常的心理模型是「强度 A = 0.5、强度 B = 0.5 就是 50/50 混合」。除非两个适配器在训练时接受了等量的梯度信号，否则这个模型是错的。96 张图训 10 轮的适配器比 67 张图训 10 轮的走得更远，相同强度并不意味着相同贡献。

因此这一对按曝光量找平训练，曝光量定义为

曝光量 = 图片数 × num_repeats × epochs

两者相差 7.5%，是取整数 repeats 之后的残差。有了这个前提，强度对 (m, f) 才构成一条可用的连续轴（见第 5 节）。

2. 模型细节

alpha 取值等于 rank 是刻意的：LoRA 缩放因子变成 alpha / rank = 1，学习率保持字面含义，不同实验之间比较时不需要心算换算。

3. 训练数据

两个数据集都是为私人研究和个人用途收集的第三方美术素材，本仓库不再分发。mretsis 是游戏 CG 截图；freestyle 是网络画师的插画作品。两批素材均未取得商业授权，仓库内不包含任何原始图片，适配器的发布目的是记录方法而非提供已授权的商业产品。使用者需自行遵守底模许可以及所在司法辖区的著作权法。

预处理。 按 1024² 面积目标分桶（enable_bucket = true、bucket_no_upscale = true）—— 不裁剪、不放大，因而构图与原生细节都被保住。latent 与文本嵌入在训练前全部预缓存（见第 4 节）。

Caption 规范。 caption 是每图一段的长自然语言描述，开头带触发词。所使用的规则与常见建议相反：

想在推理时能改的，现在必须写进 caption；想让它永远不变的，现在不能写。

caption 里点名的东西保持可被提示词操控；没点名的沉进触发词，成为适配器恒定输出的一部分。对这两个画风适配器而言：服装、姿势、背景、表情、景别都写（因此保持可覆盖），而绘画质感相关的描述不写（因此成为适配器的固定贡献）。caption 示例：

mretsis. A light-blue-haired woman in a torn black crop top ... braces one leg
forward over a body lying on the ground ... The setting is a narrow alley between
tall buildings at night. Wide shot from a low angle, her body filling the right
two thirds of the frame.

4. 训练流程

4.1 两段式预缓存是显存策略，不是可选优化

latent（VAE）与文本嵌入（Qwen3-VL-4B，约 8 GB）在训练开始前分两趟缓存到磁盘。这不是为了提速 —— 它是这次训练能在 32 GB 内不开 block swap 的原因：训练循环期间 VAE 与文本编码器都不驻留显存，全部预算留给 DiT。

需要注意的副作用：caption 一旦改动，文本嵌入缓存必须重建，否则训练会静默地继续使用旧嵌入。

4.2 超参数

accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 \
  src/musubi_tuner/krea2_train_network.py \
  --dit krea2_raw_bf16.safetensors \
  --network_module networks.lora_krea2 --network_dim 32 --network_alpha 32 \
  --learning_rate 1e-4 --optimizer_type adamw8bit \
  --timestep_sampling krea2_shift \
  --sdpa --gradient_checkpointing --fp8_base --fp8_scaled \
  --max_train_epochs 10 --save_every_n_epochs 1

训练中采样被刻意关闭。 --turbo_dit 继承 --fp8_scaled，指向已量化的 checkpoint 会崩（Layer blocks.0.attn.gate.weight is already in torch.float8_e4m3fn format）；要得到预览就得多下一个 24.5 GB 的 bf16 Turbo 模型。直接在生产用的 ComfyUI 图里评估每个存档，既更省又是更忠实的测量 —— 那才是真正的部署路径。

4.3 基础设施与实测吞吐

RTX 5090，32 GB —— 按需租用（Runpod 社区档）
2.55 s/it · batch 1 · 1024² 分桶 · fp8 + 梯度检查点
显存 24.8 / 32.6 GB（无 block swap）
mretsis   2680 步 ≈ 2 小时 00 分
freestyle 2880 步 ≈ 2 小时 09 分

5. 结果

5.1 loss 不携带可用的质量信息

<p align="center"> <img src="assets/loss_mretsis.png" width="45%" alt="mretsis 训练 loss"/> <img src="assets/loss_freestyle.png" width="45%" alt="freestyle 训练 loss"/> </p>

两次训练全程都在 0.07–0.09 之间抖动，肉眼看就是水平线。对流匹配 / 去噪目标来说这是预期行为 —— loss 主要由采样到的时间步和噪声决定，而不是由画风学得如何决定。本项目的早停与模型选择决策没有一条来自 loss 曲线，checkpoint 选择全部靠在生产图里逐轮出图对比。

5.2 收敛远早于排定的训练轮数

在 67 张图 / rank 32 / lr 1e-4 的配置下，适配器在 epoch 2（268 步）就基本收敛。loss 在 600–800 步走平，epoch 2 到 10 视觉上无法区分 —— 在 0.5 与 0.8 两个推理强度下都验证过。原本排了 10 轮，4–5 轮就够。

需要保留的前提正是这一对适配器的设计动机：打算叠加使用的适配器仍必须训练到相同曝光量，所以缩短其中一次训练就必须相应调整另一次的 num_repeats 来找平。

5.3 强度消融（5 × 5，固定提示词，固定种子）

消融扫的是 mretsis × freestyle 各取 {0.4, 0.6, 0.8, 1.0, 1.2} 的 5 × 5 网格，固定一条提示词、两个固定种子（4101 / 4102，两个种子趋势一致），共 50 张。

下图是种子 4101 在 mretsis 最低两档下、freestyle 轴完整扫过（0.4 → 1.2）的结果，其中包含日常生产配置 m0.6 + f1.0。mretsis ≥ 0.8 的行从图中略去，但没有从分析中略去：随着 mretsis 升高，它对提示词中服装描述的覆盖逐步加强——先是领口敞开，继而上衣被撑开——这本身就是 5.4 节与第 7 节报告的那个结论。图中保留的两行把轴的 freestyle 一端单独隔离出来，背景密度与装饰增生的效果在这里最清楚。

这一对构成一条连续且可解释的轴：

背景虚 · 服装简                                    背景实 · 服装华丽
   mretsis  ●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━●  freestyle
        m1.2+f0.4    m0.8+f0.8    m0.6+f1.0    m0.4+f1.2

• 偏 mretsis：背景虚化强、服装简化、人物在画面中占比更大、色调更暖 —— 来自其游戏 CG 语料的构图惯例。
• 偏 freestyle：背景细节更多、服装会自己长出装饰（绶带、金饰、边线）、穿着更整齐、更像插画。
• 这条轴是不对称的。 mretsis 的边际影响力更大：很小的权重就能把背景推虚；而 freestyle 要占到很高才能把背景推实。轴的左半段比右半段陡。

5.4 提示词控制权在哪里失效

合计强度超过 1.6 之后，提示词的风格部分基本不起作用 —— 背景一律 bokeh，明确写出的发色和服装也可能被适配器先验覆盖。这是语料视觉个性的诚实代价，属于条件控制失效，而不是渲染质量失效。

实用的解法不是降低质量预期，而是把描述搬进提示词并降低权重，让提示词和适配器说同一件事，而不是互相竞争。生产配置最终落在日常 m0.6 + f1.0、高对比 m0.8 + f0.8。

5.5 适配器先验会静默地删掉一个提示词

在一次相关消融的 248 张生成里，提示词 single side braid（单侧发辫）几乎从未被画出：两个适配器的语料都被长而蓬松的波浪发主导，这个先验压过了该词。它不是被文本编码器忽略了 —— 它是输给了适配器。与强数据集先验冲突的属性，需要比与之一致的属性多得多的提示词质量（多个互相强化的从句）才能压住。

6. 使用方法

ComfyUI，只把适配器挂到 DiT 上：

UNETLoader(krea2 ...)
  → LoraLoaderModelOnly(mretsis_krea2_v1.safetensors,  强度 0.6)
  → LoraLoaderModelOnly(freestyle_krea2_v1.safetensors, 强度 1.0)
  → KSampler(steps 12, cfg 1.0, sampler euler, scheduler simple)

触发词：mretsis. 与 freestyle.，放在提示词开头（caption 就是这样写的）。提示词用长自然语言，不要用逗号分隔的 tag —— 底模和 caption 都是围绕这一点建立的。

参考配置：m0.6 + f1.0（日常）、m0.8 + f0.8（高对比）。合计强度保持在 1.2 及以下。

另一分发渠道

完全相同的权重同时发布在 Civitai，其模型页有更多社区贡献的示例出图，也是第二条下载路径：

• freestyle_krea2_v1 · mretsis_krea2_v1

那两个页面属于面向大众的图像生成社区，含成人内容。

7. 局限与适用范围

适用范围。 研究与个人创作：复现该方法、研究流匹配图像模型的低秩适配，以及在底模许可下的非商业图像生成。

不适用范围。 商业用途；以服务形式模仿任何可识别在世画师的签名风格；以及任何会侵害原始素材权利人权益的用途。这两个适配器训练于未授权的第三方美术素材（见第 3 节），不作为已清权的商业资产提供。

已知局限。

1. 强度不是免费的 —— 合计超过 1.6 之后适配器会覆盖明确写出的提示词内容（见 5.4）。mretsis 尤其带着其源语料的暴露度惯例：约 0.8 往上就开始逐步覆盖提示词写明的服装遮蔽，到 1.2 解剖结构本身也会劣化。这些行未在消融图中展示。
2. 数据集先验会覆盖部分提示词 —— 与语料相悖的发型等属性需要不成比例的提示词质量（见 5.5）。
3. 语料偏置 —— 两个数据集绝大多数是风格化动漫 / 游戏语汇下的年轻女性形象。适配器不能泛化到其他题材，并且会带上原素材的暴露度与取景惯例。
4. 以 fp32 存储 —— 每个文件 447 MiB，约为实际所需的两倍；保存时转成 bf16 可以在无可测质量损失的前提下减半。
5. 只在单一底模上验证 —— 仅在 Krea 2 及其一个社区微调上评估过，在其他 Krea 2 衍生模型上的行为未测。
6. 仅做定性评估 —— checkpoint 与强度的选择均通过固定种子、单变量消融下的目视检查完成；未计算任何图像质量或风格相似度的定量指标，也未做人类评测。

8. 复现

仓库中的 train_lora.sh 就是实际使用的脚本，包含两趟预缓存以及以行内注释形式写下的每个参数的取舍理由。数据集不包含在内；把图片与同名 .txt caption 放进 /workspace/datasets/<名字>/ 后运行：

bash train_lora.sh style mretsis   67 4    # 2680 步，约 2 小时
bash train_lora.sh style freestyle 96 3    # 2880 步，约 2 小时