refcontrol-FLUX.2-klein-9B-reference-lineart-lora

RefControl FLUX.2 Klein 9B – Reference Lineart LoRA

📝 Short description

A LoRA for FLUX.2 Klein 9B Base that fuses a reference image (identity) with a lineart map (structure / contours).
It preserves identity and style from the reference while following the shape and composition from the lineart control map.

Trigger word: refcontrol

<Gallery />

---

📊 Examples

Each preview is a single combined image from ComfyUI: Lineart → Reference → Result (left to right).

<table> <thead> <tr> <th>Lineart → Reference → Result</th> </tr> </thead> <tbody> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-lineart-lora/resolve/main/images/a (1).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-lineart-lora/resolve/main/images/a (2).png" width="960" /></td> </tr> <tr> <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-lineart-lora/resolve/main/images/a (3).png" width="960" /></td> </tr> </tbody> </table> ---

📖 Extended description

This LoRA was primarily trained on humans, but it also works with stylized characters and some objects.
Its main purpose is to preserve identity — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the structure and contours defined by the lineart map.

FLUX.2 Klein 9B Base already handles reference + edge-guided transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and better preserves character identity and line fidelity than the base model without LoRA.

Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base.

---

⚙️ How to use

1. Use the first image as the lineart map (structure / contours).
2. Use the second image as the reference (character, person, or object).
3. Add the trigger word refcontrol in your prompt.
4. Adjust LoRA weight (recommended 0.8–1.0) depending on how strongly you want to preserve identity.

ComfyUI requirenments

Lineart extraction in the included workflow uses AnyLineArtPreprocessor_aux (via comfyui_controlnet_aux):

https://github.com/Fannovel16/comfyui_controlnet_aux

You can disable the built-in lineart preprocessor if your input is already a lineart/edge image.

Base model

Trained on and recommended with black-forest-labs/FLUX.2-klein-base-9B.
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, guidance_scale ~4.0).

The LoRA also works with the 4-step distilled black-forest-labs/FLUX.2-klein-9B for faster inference (~4 steps, guidance_scale ~1.0), but quality may be slightly lower — especially for identity and line fidelity — compared to the Base model.

✅ Example prompt

refcontrol

---

🎯 What it does

• Preserves character or object identity across generations.
• Adapts the subject to a new structure or composition defined by the lineart map.
• Works best when the lineart map has similar proportions and scale to the reference.

---

⚡ Tips

• Best results when the lineart map is not drastically different in body scale or framing from the reference.
• Combine with text prompts to refine background, lighting, or mood.
• Lineart maps on a white or light background work well as control input.

---

📌 Use cases

• Character restyling while keeping identity and line-defined structure.
• Consistent character design across different compositions.
• Illustration and storyboard generation with line-guided layout.
• Object transformations with contour-guided placement.

---

📦 Files

• Weights: flux2_klein_9b_refcontrol_lineart.safetensors
• ComfyUI workflow: refcontrol_lineart_flux_klein_9b.json
• Repo: thedeoxen/refcontrol-FLUX.2-klein-9B-reference-lineart-lora