wan21-fp16-loras

<!-- README Version: v1.3 -->

WAN 2.1 Camera Control LoRAs (FP16)

Advanced camera motion control LoRA adapters for WAN (World Animation Network) video generation models. These rank-16 LoRAs enable precise control over camera movements including rotation, arc shots, and drone-style cinematography.

Model Description

This repository contains three specialized LoRA adapters designed to enhance video generation with professional camera movement patterns:

• Camera Rotation: Enables smooth 360° orbital camera movements around subjects
• Arc Shot: Creates cinematic arc/dolly movements for dynamic scene transitions
• Drone Shot: Simulates aerial drone cinematography with elevation and forward motion

These LoRAs are trained at rank-16 for optimal balance between parameter efficiency and motion quality control. All models are stored in FP16 precision for maximum compatibility with standard diffusion pipelines.

Repository Contents

wan21-fp16-loras/
└── loras/
    └── wan/
        ├── wan21-camera-rotation-rank16-v1.safetensors    (343 MB)
        ├── wan21-camera-arcshot-rank16-v1.safetensors     (343 MB)
        └── wan21-camera-drone-rank16-v1.safetensors       (343 MB)

Total Repository Size: ~1.1 GB

Hardware Requirements

Minimum Requirements

• VRAM: 8 GB (for inference with base model)
• RAM: 16 GB system memory
• Disk Space: 1.1 GB for LoRAs + base model requirements
• GPU: NVIDIA GPU with CUDA support (RTX 3060 or better recommended)

Recommended Requirements

• VRAM: 16 GB or higher for optimal performance
• RAM: 32 GB system memory
• Disk Space: 10 GB+ for models and output videos
• GPU: NVIDIA RTX 4080/4090 or A100 for best performance

Usage Examples

Basic Usage with Diffusers

import torch
from diffusers import DiffusionPipeline

# Load base WAN model
pipe = DiffusionPipeline.from_pretrained(
    "HunyuanVideo/HunyuanVideo",
    torch_dtype=torch.float16,
    variant="fp16"
)
pipe.to("cuda")

# Load camera rotation LoRA
pipe.load_lora_weights(
    "E:\\huggingface\\wan21-fp16-loras\\loras\\wan",
    weight_name="wan21-camera-rotation-rank16-v1.safetensors"
)

# Generate video with camera rotation
prompt = "A majestic lion sitting on a rock, cinematic lighting, 4k"
video = pipe(
    prompt=prompt,
    num_frames=48,
    height=512,
    width=512,
    num_inference_steps=50,
    guidance_scale=7.5,
    cross_attention_kwargs={"scale": 0.8}  # LoRA strength
).frames

# Save video
from diffusers.utils import export_to_video
export_to_video(video, "output_rotation.mp4", fps=8)

Switching Between Camera LoRAs

# Unload current LoRA
pipe.unload_lora_weights()

# Load arc shot LoRA
pipe.load_lora_weights(
    "E:\\huggingface\\wan21-fp16-loras\\loras\\wan",
    weight_name="wan21-camera-arcshot-rank16-v1.safetensors"
)

# Generate with arc shot movement
video = pipe(
    prompt="A bustling city street at sunset, cinematic arc shot",
    num_frames=48,
    cross_attention_kwargs={"scale": 0.7}
).frames

Adjusting LoRA Strength

# Subtle camera movement (scale: 0.3-0.5)
video = pipe(
    prompt="Static scene with subtle camera drift",
    cross_attention_kwargs={"scale": 0.4}
).frames

# Standard camera movement (scale: 0.6-0.8)
video = pipe(
    prompt="Dynamic scene with smooth camera motion",
    cross_attention_kwargs={"scale": 0.7}
).frames

# Dramatic camera movement (scale: 0.9-1.0)
video = pipe(
    prompt="Action scene with aggressive camera work",
    cross_attention_kwargs={"scale": 1.0}
).frames

Drone Shot Example

pipe.unload_lora_weights()
pipe.load_lora_weights(
    "E:\\huggingface\\wan21-fp16-loras\\loras\\wan",
    weight_name="wan21-camera-drone-rank16-v1.safetensors"
)

# Generate aerial drone footage
video = pipe(
    prompt="Aerial view of a mountain valley, rising drone shot, golden hour",
    num_frames=64,
    height=768,
    width=1344,
    cross_attention_kwargs={"scale": 0.8}
).frames

export_to_video(video, "drone_shot.mp4", fps=12)

Model Specifications

Architecture

• Type: LoRA (Low-Rank Adaptation) adapters
• Rank: 16
• Target Modules: Cross-attention layers in temporal transformer blocks
• Precision: FP16 (Float16)
• Format: SafeTensors (.safetensors)
• Base Model: Compatible with WAN/HunyuanVideo architecture

Training Details

• Version: v1 (Initial release)
• Training Data: Curated video datasets with professional camera movements
• Optimization: Camera motion quality and temporal consistency
• Specialization: Each LoRA trained on specific camera movement patterns

Camera Movement Characteristics

Rotation LoRA:

• 360° orbital movements around central subject
• Maintains consistent distance and elevation
• Smooth, continuous rotation speed
• Best for: Product showcases, character reveals, architectural tours

Arc Shot LoRA:

• Curved dolly movements (lateral + forward/backward)
• Dynamic perspective shifts
• Cinematic scene transitions
• Best for: Dramatic reveals, environmental storytelling, action sequences

Drone Shot LoRA:

• Aerial perspective with elevation changes
• Forward motion with ascending/descending
• Wide establishing shots
• Best for: Landscape videos, establishing shots, bird's-eye views

Performance Tips and Optimization

LoRA Strength Guidelines

• Start with scale=0.7 as baseline for most scenes
• Reduce to 0.3-0.5 for subtle, naturalistic camera work
• Increase to 0.9-1.0 for dramatic, stylized movements
• Experiment with different scales to match your creative vision

Quality Optimization

• Use higher resolution (768x1344 or 1024x1024) for smoother motion
• Increase num_inference_steps to 60-80 for better quality
• Generate more frames (64-96) for longer, smoother sequences
• Use guidance_scale 7-9 for balanced prompt adherence

Memory Optimization

• Use torch.float16 precision for inference
• Enable attention slicing: pipe.enable_attention_slicing()
• Enable VAE tiling: pipe.enable_vae_tiling()
• Reduce resolution or frame count if VRAM limited

Batch Processing

# Process multiple prompts efficiently
prompts = [
    "Mountain landscape, drone rising shot",
    "City street, rotating camera view",
    "Forest scene, cinematic arc shot"
]

for i, prompt in enumerate(prompts):
    video = pipe(prompt=prompt, cross_attention_kwargs={"scale": 0.7}).frames
    export_to_video(video, f"output_{i}.mp4")

Combining with Other Enhancements

# Load multiple LoRAs (if supported by pipeline)
pipe.load_lora_weights("path/to/style_lora.safetensors", adapter_name="style")
pipe.load_lora_weights(
    "E:\\huggingface\\wan21-fp16-loras\\loras\\wan\\wan21-camera-rotation-rank16-v1.safetensors",
    adapter_name="camera"
)

# Set adapter weights
pipe.set_adapters(["style", "camera"], adapter_weights=[0.5, 0.7])

License

These LoRA models are subject to the WAN license terms. Please review the license agreement before commercial use:

• Research Use: Permitted with proper attribution
• Commercial Use: May require separate licensing agreement
• Distribution: Allowed with original license documentation
• Modification: Permitted for research and personal projects

For commercial licensing inquiries, please contact the original model creators or refer to the base model repository.

Citation

If you use these LoRAs in your research or projects, please cite:

@misc{wan21-camera-loras,
  title={WAN 2.1 Camera Control LoRAs},
  author={WAN Development Team},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/HunyuanVideo/WAN-LoRAs}}
}

Additional Resources

• Base Model: HunyuanVideo/HunyuanVideo
• Diffusers Documentation: huggingface.co/docs/diffusers
• LoRA Guide: Hugging Face LoRA Documentation
• Community Forum: Hugging Face Discussion Boards

Troubleshooting

Common Issues

Issue: "Out of memory" error during inference

• Solution: Enable memory optimizations, reduce resolution/frames, or use lower LoRA scale

Issue: Camera movement too subtle or not visible

• Solution: Increase LoRA scale parameter (try 0.9-1.0)

Issue: Unrealistic or jerky camera motion

• Solution: Reduce LoRA strength, increase inference steps, use more frames

Issue: LoRA not loading correctly

• Solution: Verify file path, ensure base model compatibility, check diffusers version

Support

For technical issues, bug reports, or feature requests:

• Open an issue on the model repository
• Check existing discussions and documentation
• Ensure you're using compatible versions of diffusers and PyTorch

---

Model Version: v1.0 Last Updated: October 2024 Repository Maintained By: WAN Development Team