aetherart-pattachitra-sdxl

AetherArt — SDXL Pattachitra LoRA

Rank-8 LoRA adapter for Pattachitra (traditional Odisha folk art) style transfer on stabilityai/stable-diffusion-xl-base-1.0, trained on a 100-image curated corpus. Activate the style with the trigger token pattascroll anywhere in the prompt.

Recommended usage: checkpoint-1000 at adapter weight 0.5, NOT the library default of 1.0

Default configuration: checkpoint-1000 at adapter_weights=[0.5] — this is the configuration the published sample images below were generated at, and the one shown in Usage. At the library-default full weight (1.0), this adapter measurably REGRESSES both style authenticity and figure/subject preservation relative to sdxl_base with no adapter at all — do not deploy at weight 1.0.

Intermediate weights between 0.3 and 0.5 were not individually tested — the recommendation is a range bounded by the two tested points that both pass the joint style/figure criterion, not a claim that every value in between has been independently verified.

Sample outputs (checkpoint-1000, weight=0.5, seed 42)

"pattascroll Pattachitra painting of Lord Jagannath, Balabhadra, and Subhadra in a temple shrine"

"pattascroll Pattachitra scroll painting of Krishna playing the flute under a tree"

"pattascroll Pattachitra painting of the goddess Durga slaying a demon"

Usage

from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
    variant="fp16",
    use_safetensors=True,
)
pipe.load_lora_weights(
    "gauravgandhi2411/aetherart-pattachitra-sdxl",
    weight_name="checkpoint-1000/pytorch_lora_weights.safetensors",
    adapter_name="pattachitra",
)
# Recommended: 0.3-0.5. Do NOT use the library default of 1.0 (see table above).
pipe.set_adapters(["pattachitra"], adapter_weights=[0.5])
pipe.enable_model_cpu_offload()  # required on GPUs < 16 GB VRAM

img = pipe(
    "pattascroll Pattachitra painting of Lord Jagannath, Balabhadra, and Subhadra in a temple shrine",
    negative_prompt="text, watermark, caption, museum label, calligraphy, signature, words, letters",
    num_inference_steps=30,
    guidance_scale=7.5,
).images[0]
img.save("output.png")

Alternatives

checkpoint-500 is also published (checkpoint-500/pytorch_lora_weights.safetensors) for a lighter style application. Evaluation found no significant difference between the two checkpoints at low weight — style_adherence was statistically identical at weight 0.5 (+3.482×SEM, both checkpoints), and figure_preservation was comparable (checkpoint-500 +2.080×SEM vs. checkpoint-1000 +1.422×SEM at weight 0.5 — checkpoint-500 numerically higher, not established as a meaningfully different result at this sample size). checkpoint-1000 is the default shown above because it is what the published sample images were generated with, not because evaluation demonstrated it as superior. To use checkpoint-500 instead, pass weight_name="checkpoint-500/pytorch_lora_weights.safetensors" — checkpoint-500 also has its own confirmed operating point at weight 0.3 (style_adherence +3.622×SEM, the highest single value measured across both checkpoints).

Known limitations

• Full-weight figure/subject dropout: at weight=1.0, generations can drop prompted human figures entirely (visually confirmed: identical prompt+seed vs. sdxl_base, the LoRA output omits the person, sdxl_base renders them correctly). This is a checkpoint-wide effect (500 and 1000 steps both affected, likely stemming from generic auto-generated training captions), not fixed by choosing a different checkpoint — it is fixed by choosing a lower weight.

• Prompt-fidelity limitation, even at the recommended weight: the sample below was prompted for "Hanuman carrying a mountain of herbs" — the central figure instead reads as an elephant-headed deity (Ganesha-like iconography), not the prompted monkey-deity Hanuman. Published uncurated, as an honest example of a real failure mode rather than an unexplained gap in the gallery above.

  "pattascroll Pattachitra painting of Hanuman carrying a mountain of herbs" — checkpoint-1000, weight=0.5, seed 42

Measurement methodology

Independent-axis VLM judge (qwen2.5vl:7b, zero-cost local), n=90 paired per weight/checkpoint comparison vs. sdxl_base, adapter-weight sweep at 0.3/0.5/0.7/1.0. A positive control confirmed the judge can reliably perceive Pattachitra style (PASS, decisively, against two independent off-style contrasts — real ukiyo-e art and generic non-styled sdxl_base outputs, diff/SEM = +17.951 and +28.659 respectively, both far exceeding their own minimum detectable effect). Full methodology and complete numbers: AetherArt GitHub repo, docs/MODEL_VERDICT.md §7.

Training details

Default negative prompt

text, watermark, caption, museum label, calligraphy, signature, words, letters

Links

• AetherArt repository: https://github.com/gaurav-gandhi-2411/AetherArt
• Full evaluation methodology and numbers: docs/MODEL_VERDICT.md §7 in the AetherArt GitHub repo
• Companion ukiyo-e adapters: gauravgandhi2411/aetherart-ukiyo-sdxl, gauravgandhi2411/aetherart-ukiyo-sd21