wan22-fp8-t2v-loras

<!-- README Version: v1.4 -->

WAN LoRA Collection v2.2 - Text-to-Video Edition (FP8)

A curated collection of LoRA adapters for the WAN (Wangfuyun Animate) video generation model, providing enhanced camera control, lighting effects, and quality improvements specifically optimized for text-to-video (T2V) generation with FP8 precision.

Model Description

This repository contains 8 specialized LoRA adapters designed to enhance WAN video generation capabilities:

• Camera Control LoRAs: 5 models for precise camera movement control (advanced aerial shots, arc movements, rotation, drone perspective, earth zoom-out)
• Lighting LoRAs: 1 model for cinematic volumetric lighting effects
• Quality Enhancement LoRAs: 2 models for improved realism, facial naturalization, and upscaling

These LoRAs are designed to work with WAN base models (14B parameters) and provide fine-grained control over video generation parameters while maintaining compatibility with FP8 precision models for efficient inference.

Repository Contents

wan22-fp8-t2v-loras/
└── loras/
    └── wan/
        ├── wan22-camera-adr1a-t2v-14b.safetensors            (293 MB)
        ├── wan22-camera-arcshot-rank16-v2-high.safetensors  (293 MB)
        ├── wan22-camera-drone-rank16-v2.safetensors         (293 MB)
        ├── wan22-camera-earthzoomout.safetensors            (293 MB)
        ├── wan22-camera-rotation-rank16-v2.safetensors      (293 MB)
        ├── wan22-face-naturalizer-t2v.safetensors           (586 MB)
        ├── wan22-light-volumetric-t2v.safetensors           (293 MB)
        └── wan22-upscale-realismboost-t2v-14b.safetensors   (293 MB)

Total Repository Size: 2.6 GB

LoRA Descriptions

Camera Control LoRAs:

• wan22-camera-adr1a-t2v-14b: Advanced camera control with ADR1A algorithm for 14B T2V model
• wan22-camera-arcshot-rank16-v2-high: High-quality arc shot movements (rank-16, v2 optimized)
• wan22-camera-drone-rank16-v2: Aerial drone-style perspective shots (rank-16, v2 optimized)
• wan22-camera-earthzoomout: Dramatic earth zoom-out effect for cosmic-scale scenes
• wan22-camera-rotation-rank16-v2: Circular rotation around subject (rank-16, v2 optimized)

Lighting LoRAs:

• wan22-light-volumetric-t2v: Volumetric lighting effects optimized for T2V (god rays, atmospheric depth, cinematic lighting)

Quality Enhancement LoRAs:

• wan22-face-naturalizer-t2v: Natural facial appearance and movement enhancement for T2V (586 MB, higher capacity)
• wan22-upscale-realismboost-t2v-14b: Realism enhancement for T2V generation (14B model compatible)

Hardware Requirements

Minimum Requirements

• VRAM: 16 GB (for base WAN model + 1-2 LoRAs)
• System RAM: 32 GB
• Disk Space: 2.6 GB (for LoRA collection only)
• GPU: NVIDIA RTX 3090 / 4090 or equivalent

Recommended Requirements

• VRAM: 24 GB or higher (RTX 4090, A5000, or better)
• System RAM: 64 GB
• Disk Space: 40+ GB (including base WAN models and video outputs)
• GPU: NVIDIA RTX 4090, A6000, or higher

Note: LoRAs have minimal VRAM overhead (~300-600 MB each) but require a base WAN model loaded in memory (~8-12 GB for FP8 models). These T2V-optimized LoRAs work best with WAN 14B parameter models.

Usage Examples

Loading with Diffusers (Python)

from diffusers import DiffusionPipeline
import torch

# Load base WAN model
pipe = DiffusionPipeline.from_pretrained(
    "wangfuyun/AnimateLCM-SVD-xt",
    torch_dtype=torch.float8_e4m3fn,
    variant="fp8"
)
pipe.to("cuda")

# Load camera rotation LoRA
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp8-t2v-loras/loras/wan",
    weight_name="wan22-camera-rotation-rank16-v2.safetensors",
    adapter_name="camera_rotation"
)

# Set LoRA strength (0.0-1.0)
pipe.set_adapters(["camera_rotation"], adapter_weights=[0.8])

# Generate video with camera rotation
prompt = "A beautiful mountain landscape at sunset"
video = pipe(
    prompt=prompt,
    num_frames=24,
    guidance_scale=7.5,
    num_inference_steps=25
).frames

# Save video
from diffusers.utils import export_to_video
export_to_video(video, "output_rotation.mp4", fps=8)

Combining Multiple LoRAs

# Load multiple LoRAs for combined effects
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp8-t2v-loras/loras/wan",
    weight_name="wan22-camera-drone-rank16-v2.safetensors",
    adapter_name="drone_camera"
)

pipe.load_lora_weights(
    "E:/huggingface/wan22-fp8-t2v-loras/loras/wan",
    weight_name="wan22-light-volumetric-t2v.safetensors",
    adapter_name="volumetric_light"
)

# Set multiple adapters with individual weights
pipe.set_adapters(
    ["drone_camera", "volumetric_light"],
    adapter_weights=[0.7, 0.5]
)

# Generate video with drone camera + volumetric lighting
video = pipe(
    prompt="Ancient temple in misty jungle with sunlight filtering through trees",
    num_frames=32,
    guidance_scale=8.0
).frames

Quality Enhancement for Text-to-Video

# Load realism boost and face naturalizer LoRAs for T2V
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp8-t2v-loras/loras/wan",
    weight_name="wan22-upscale-realismboost-t2v-14b.safetensors",
    adapter_name="realism_boost"
)

pipe.load_lora_weights(
    "E:/huggingface/wan22-fp8-t2v-loras/loras/wan",
    weight_name="wan22-face-naturalizer-t2v.safetensors",
    adapter_name="face_natural"
)

# Combine quality enhancement LoRAs
pipe.set_adapters(
    ["realism_boost", "face_natural"],
    adapter_weights=[0.75, 0.65]
)

# Generate high-quality realistic video with natural faces
video = pipe(
    prompt="Photorealistic close-up portrait of a person smiling naturally",
    num_frames=48,
    guidance_scale=9.0,
    num_inference_steps=30
).frames

export_to_video(video, "output_quality.mp4", fps=8)

Model Specifications

Technical Details

• Format: SafeTensors (secure, efficient binary format)
• Precision: FP8-compatible (designed for FP8 base models)
• Architecture: LoRA (Low-Rank Adaptation) adapters
• Rank: Mostly rank-16 for efficiency (some higher capacity models)
• Compatibility: WAN/AnimateLCM-SVD-xt base models

LoRA Strengths

• Low Noise Variants: Recommended strength 0.6-0.9
• Standard Models: Recommended strength 0.5-0.8
• High-Quality Models: Recommended strength 0.7-1.0

Combination Guidelines:

• Camera + Lighting: Safe to combine (reduce each to 0.6-0.7)
• Multiple Camera LoRAs: Not recommended (conflicting movements)
• Quality Enhancement (Realism + Face): Excellent combination for human subjects
• Camera + Quality: Works well together for cinematic results

Performance Tips

1. LoRA Selection: Use 1-3 LoRAs per generation (camera, lighting, quality work well together)
2. Strength Tuning: Start with 0.7 and adjust based on desired effect intensity
3. Camera Control: Use one camera LoRA at a time for consistent movement
4. Lighting Effects: Volumetric lighting combines well with camera control for cinematic results
5. Quality Enhancement: Combine realism boost + face naturalizer for human-focused content
6. T2V Optimization: All LoRAs in this collection are optimized for text-to-video generation
7. FP8 Precision: These LoRAs are optimized for FP8 inference (faster, lower VRAM)
8. 14B Model: Best results with WAN 14B parameter models (as indicated by -14b suffixes)
9. Batch Processing: Process multiple videos with same LoRA loaded (amortize loading time)

Optimization Recommendations

# Enable memory optimization
pipe.enable_attention_slicing()
pipe.enable_vae_slicing()

# Use FP16 for intermediate calculations (FP8 for weights)
pipe.unet.to(dtype=torch.float16)
pipe.vae.to(dtype=torch.float16)

# Enable xformers for faster attention (if installed)
pipe.enable_xformers_memory_efficient_attention()

License

This LoRA collection follows the WAN (Wangfuyun Animate) license terms. Please refer to the base model license at: https://huggingface.co/wangfuyun/AnimateLCM-SVD-xt

Usage Restrictions:

• Free for research and non-commercial use
• Commercial use may require permission from original authors
• Derivative works should credit the original WAN model and LoRA creators
• Do not use for generating harmful, illegal, or unethical content

Citation

If you use these LoRAs in your research or projects, please cite:

@misc{wan22-lora-collection,
  title={WAN LoRA Collection v2.2: Enhanced Camera Control and Quality for Video Generation},
  author={Wangfuyun AnimateLCM Team},
  year={2024},
  howpublished={\url{https://huggingface.co/wangfuyun/AnimateLCM-SVD-xt}},
  note={LoRA adapters for camera control, lighting, and quality enhancement}
}

Resources

• Base Model: wangfuyun/AnimateLCM-SVD-xt
• Diffusers Documentation: https://huggingface.co/docs/diffusers
• LoRA Training Guide: https://huggingface.co/docs/diffusers/training/lora
• WAN Community: Hugging Face Discussions

Changelog

Version 1.0 (2024-10-13)

• Initial comprehensive README with Hugging Face metadata
• Documentation for all LoRA adapters
• Usage examples for camera control, lighting, and quality enhancement
• Hardware requirements and optimization tips
• Multi-LoRA combination guidelines

Version 1.1 (2024-10-14)

• Fixed YAML frontmatter position (moved to line 1 as required)
• Updated version header placement (now after YAML frontmatter)

Version 1.2 (2025-10-14)

• Verified YAML metadata compliance with HuggingFace standards
• Validated all technical specifications and file sizes
• Updated version tracking and changelog

Version 1.3 (2025-10-14)

• Updated to reflect actual repository contents (8 T2V-optimized LoRAs)
• Removed I2V-specific examples (this collection is T2V-focused)
• Corrected file names and sizes based on actual directory contents
• Updated total repository size (2.6 GB)
• Enhanced T2V-specific usage guidance and optimization tips
• Clarified 14B model compatibility throughout documentation

Version 1.4 (2025-10-28)

• Simplified YAML frontmatter to meet HuggingFace requirements
• Removed non-standard metadata fields (license_name, license_link)
• Streamlined tags to core identifiers (wan, text-to-video, image-generation)
• Validated YAML structure for proper HuggingFace compatibility

---

Model Version: v2.2 README Version: v1.4 Last Updated: 2025-10-28 Repository Size: 2.6 GB