animate-lora-sd1.5

animate-lora-sd1.5

LoRA adapter for cinematic anime-style landscape generation on top of Stable Diffusion 1.5.

Model summary

• Base model: runwayml/stable-diffusion-v1-5
• Trigger words: landscape, sms landscape
• Adapter file: animate_v1-000005.safetensors
• Intended style: cinematic anime-style scenery, sky-rich composition, stylized background art

Intended use

This adapter is intended for stylized landscape generation, scenic diary illustrations, and anime-inspired background imagery.

It works best as a style adapter layered on top of SD1.5 rather than as a broad general-purpose object model.

Related project

• Companion app repo: https://github.com/J-Y00N/Multimodal-Picture-Diary
• Live demo page: https://multimodal-picture-diary-25vtwte77q6nxwtbdhhpp9.streamlit.app/

This model repo is maintained by the same author as the companion app, but published separately so the LoRA release and the application code can be versioned independently.

Preserved training evidence

Preserved local artifacts suggest the original training run used:

• resolution: 512x512
• network_dim: 25
• network_alpha: 25
• train_batch_size: 16
• text_encoder_lr: 5e-05
• unet_lr: 0.0001
• optimizer: AdamW
• max_train_steps: 93750
• lr_warmup_steps: 9375
• xformers enabled
• dataset tag frequencies preserved in metadata/animation_training_log.txt

Preserved configs and logs indicate a kohya_ss-based LoRA training workflow.

See metadata/painting_v1_20231211-043052.json, metadata/painting_script_caption.txt, and metadata/animation_training_log.txt for the preserved config snapshot and training notes.

Dataset reference

• Related public dataset reference: Hugging Face - Fung804/makoto-shinkai-picture

This link is included as a public dataset reference related to the preserved landscape/anime-style training artifacts. The exact one-to-one mapping to the released checkpoint is not fully guaranteed.

Known unknowns

• The exact mapping between the preserved local painting_v1 artifacts and this public checkpoint is not fully guaranteed.
• Exact seed and exact dataset snapshot were not preserved.
• This repository is an inference-oriented adapter release, not a full archival dump of the original training environment.

Diffusers usage

import torch
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16,
).to("cuda")

pipe.load_lora_weights(
    "J-YOON/animate-lora-sd1.5",
    weight_name="animate_v1-000005.safetensors",
)

prompt = "sms landscape, evening sky over a quiet city, cinematic diary illustration"
image = pipe(prompt, num_inference_steps=30, guidance_scale=7.0).images[0]
image.save("animate_example.png")

Limitations

• The adapter is specialized for landscape-oriented and background-like imagery.
• Non-landscape prompts may be biased back toward scenic composition or stylized atmosphere.
• For broader object coverage, reduce adapter strength or fall back to the base model when needed.

Gallery

<table> <tr> <td align="center"><strong>Generated: Dog</strong></td> <td align="center"><strong>Generated: Diary img2img</strong></td> </tr> <tr> <td><img src="./images/example_dog_animate.png" alt="Animate dog example" width="260" /></td> <td><img src="./images/example_rainy_walk_animate.png" alt="Diary app animate img2img example" width="260" /></td> </tr> <tr> <td align="center"><strong>Reference: Dog</strong></td> <td align="center"><strong>Reference: Rainy walk</strong></td> </tr> <tr> <td><img src="./images/reference_dog_photo.jpg" alt="Reference dog photo" width="260" /></td> <td><img src="./images/reference_rainy_walk_photo.jpg" alt="Reference rainy walk photo" width="260" /></td> </tr> </table>