art-lora-flux1-dev

art-lora-flux1-dev

Style LoRA for FLUX.1-dev. Trained on 19 curated painterly character-design references.

Trigger: sk3tchref

Hand-painted character concept art — broad flat brush shapes, hard-edged colour blocking, minimal blending, muted olive / charcoal / cream / tan palette, flat off-white background with a thin shadow smear, stylized elongated proportions.

---

Repo contents

Style LoRAs overfit, so the best checkpoint is not always the last. All of them are here so you can compare — see samples/_grids/.

---

1. Download the models

Everything below goes into your ComfyUI models/ tree. Total ~34 GB.

cd /workspace/ComfyUI      # adjust to your ComfyUI root
mkdir -p models/diffusion_models models/text_encoders models/vae models/loras

Transformer — FLUX.1-dev, 23.8 GB

The official repo black-forest-labs/FLUX.1-dev is gated: you must accept the licence on the model page and use a read token. Same for FLUX.1-schnell.

# Option A — official, gated. Accept the licence first at
#   https://huggingface.co/black-forest-labs/FLUX.1-dev
# then create a READ token at https://huggingface.co/settings/tokens
export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxx

wget --header="Authorization: Bearer $HF_TOKEN" \
  -O models/diffusion_models/flux1-dev.safetensors \
  "https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/flux1-dev.safetensors"

# Option B — Comfy-Org mirror. Identical weights, no token needed at time of writing.
wget -O models/diffusion_models/flux1-dev.safetensors \
  "https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev.safetensors"

Option B is what this LoRA was trained against. Mirrors can be gated or removed without notice — if Option B starts returning 401, fall back to Option A.

Text encoders — 10 GB

wget -O models/text_encoders/t5xxl_fp16.safetensors \
  "https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors"

wget -O models/text_encoders/clip_l.safetensors \
  "https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors"

VAE — 335 MB

# ungated mirror (black-forest-labs/FLUX.1-schnell is gated and returns 401)
wget -O models/vae/ae.safetensors \
  "https://huggingface.co/ffxvs/vae-flux/resolve/main/ae.safetensors"

This LoRA

wget -O models/loras/sk3tchref.safetensors \
  "https://huggingface.co/kirusanth08/art-lora-flux1-dev/resolve/main/sk3tchref.safetensors"

# intermediate checkpoints, if you want to compare
for S in 00000500 00001000 00001500 00002000; do
  wget -O "models/loras/sk3tchref-step$S.safetensors" \
    "https://huggingface.co/kirusanth08/art-lora-flux1-dev/resolve/main/sk3tchref-step$S.safetensors"
done

If the repo is private, add --header="Authorization: Bearer $HF_TOKEN" to those too.

Verify

A truncated download does not error — it fails later at load time with 'NoneType' object has no attribute 'Params'. Check the sizes:

ls -l models/diffusion_models/flux1-dev.safetensors   # 23802932552
ls -l models/text_encoders/t5xxl_fp16.safetensors     #  9787841024
ls -l models/text_encoders/clip_l.safetensors         #   246144152
ls -l models/vae/ae.safetensors                       #   335304388

---

2. Run it

wget -O workflow.json \
  "https://huggingface.co/kirusanth08/art-lora-flux1-dev/resolve/main/workflow.json"

Drag workflow.json onto the ComfyUI canvas. It wires up:

UNETLoader (flux1-dev, fp8_e4m3fn)
  └─ LoraLoaderModelOnly (sk3tchref.safetensors, strength 1.0)
       └─ KSampler ── VAEDecode ── SaveImage
DualCLIPLoader (clip_l + t5xxl_fp16, type "flux")
  └─ CLIPTextEncode ─┬─ FluxGuidance 3.0 ──→ positive
                     └─ ConditioningZeroOut ─→ negative

workflow_api.json is the same graph for POST /prompt.

Settings that matter

cfg must stay at 1.0. Flux dev is CFG-distilled — raising it destroys the image, and negative prompts do nothing, which is why the negative branch is just a zeroed-out copy of the positive. FluxGuidance (~3.0) is the real prompt-adherence knob.

Sampler euler, scheduler beta, 28 steps, 832×1216.

Prompt template

sk3tchref, a full body character illustration of <SUBJECT>, hand-painted character
concept art, broad flat brush shapes, hard-edged colour blocking, muted desaturated
palette, flat off-white background, thin grey shadow smear under the feet

The trigger must come first.

Strength guide

See samples/final/sweep.jpg.

---

3. How it was trained

kohya sd-scripts (sd3 branch), flux_train_network.py, on an RTX 5090 32 GB. ~1.75 s/it, ~1 hour for 2000 steps.

accelerate launch --num_processes 1 --mixed_precision bf16 \
  --num_cpu_threads_per_process 8 flux_train_network.py \
  --pretrained_model_name_or_path $M/diffusion_models/flux1-dev.safetensors \
  --clip_l $M/text_encoders/clip_l.safetensors \
  --t5xxl $M/text_encoders/t5xxl_fp16.safetensors \
  --ae $M/vae/ae.safetensors \
  --dataset_config dataset.toml \
  --output_dir ./out --output_name sk3tchref \
  --save_model_as safetensors --save_precision bf16 \
  --network_module networks.lora_flux --network_dim 24 --network_alpha 24 \
  --network_train_unet_only \
  --optimizer_type adamw8bit --learning_rate 1e-4 \
  --lr_scheduler constant_with_warmup --lr_warmup_steps 40 \
  --max_train_steps 2000 --save_every_n_steps 500 \
  --gradient_checkpointing --mixed_precision bf16 --fp8_base --sdpa --highvram \
  --cache_latents_to_disk --cache_text_encoder_outputs --cache_text_encoder_outputs_to_disk \
  --max_data_loader_n_workers 2 --persistent_data_loader_workers --seed 42 \
  --timestep_sampling shift --discrete_flow_shift 3.1582 \
  --model_prediction_type raw --guidance_scale 1.0 --loss_type l2

Dataset config: aspect-ratio bucketing at 1024 base, min_bucket_reso 512, max_bucket_reso 1536, keep_tokens = 1, shuffle_caption = false.

---

4. The captioning method

This is the part that decides whether a style LoRA works.

Captions describe content only — who the character is, what they wear, what they do:

sk3tchref, two views, front and back, of a heavy-set older man with a grey beard in an
olive green jacket, cream shirt and dark green trousers, holding a bottle

Note what is absent: no "painterly", no "flat colour blocking", no "muted palette", no "white background". Everything constant across the set and missing from the captions collapses onto the single token they all share — sk3tchref.

Write style words into the captions and the style scatters across those common words instead, producing a LoRA that is weak, hard to trigger, and unresponsive to strength. keep_tokens = 1 pins the trigger at the front so shuffling never displaces it.

Every caption in dataset/ was written by looking at the image.

---

Why a LoRA and not --sref

Reference-image conditioning was tested first and does not work for style transfer:

Conditioning transfers content. Prompting transfers the base model's prior. Only weight updates transfer style.