Wan2.1-1.3b-lora-speedcontrol-v1

Wanxiao 2.1-1.3B-LoRA-Speed-Control-v1

Model Introduction

This LoRA model is trained based on the Wanxiao 2.1-1.3B model and the DiffSynth-Studio framework. This model allows control over video speed by adjusting the LoRA alpha parameter.

• LoRA alpha > 0: Use the trigger word "low speed" to slow down the video speed and enhance visual quality.
• LoRA alpha < 0: Use the trigger word "high speed" to speed up the video and reduce visual quality.

Model Performance

Prompt: Documentary photography style, a lively white puppy rapidly running on a lush green lawn. The puppy has pure white fur, erect ears, and an expression of focused joy. Sunlight shines on its body, making the fur appear exceptionally soft and shiny. The background features an open grassland dotted with occasional wildflowers, with a faint view of blue sky and scattered clouds in the distance. Strong perspective emphasizes the dynamic motion of the running puppy and the vitality of the surrounding grass. Medium shot with a side-moving viewpoint.

Negative prompt: Vivid colors, overexposure, static, blurry details, subtitles, style, artwork, painting, stillness, overall grayish tone, worst quality, low quality, JPEG compression artifacts, ugly, defective, extra fingers, poorly drawn hands, poorly drawn face, deformed limbs, fused fingers, motionless frames, cluttered background, three legs, crowded background, walking backward.

LoRA alpha = 0.7

<div align="center"><video width="80%" controls><source src="assets/0.7.mp4" type="video/mp4">Your browser does not support the video tag.</video></div>

LoRA alpha = 0

<div align="center"><video width="80%" controls><source src="assets/0.mp4" type="video/mp4">Your browser does not support the video tag.</video></div>

LoRA alpha = -0.5

<div align="center"><video width="80%" controls><source src="assets/-0.5.mp4" type="video/mp4">Your browser does not support the video tag.</video></div>

Usage Instructions

This model is built on the DiffSynth-Studio framework. Please install it first:

pip install diffsynth

import torch
from diffsynth import ModelManager, WanVideoPipeline, save_video
from modelscope import snapshot_download

snapshot_download(
    model_id="DiffSynth-Studio/Wan2.1-1.3b-lora-speedcontrol-v1",
    local_dir="models/DiffSynth-Studio/Wan2.1-1.3b-lora-speedcontrol-v1",
    allow_file_pattern="*.safetensors"
)
model_manager = ModelManager(device="cpu")
model_manager.load_models(
    [
        "models/Wan-AI/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors",
        "models/Wan-AI/Wan2.1-T2V-1.3B/models_t5_umt5-xxl-enc-bf16.pth",
        "models/Wan-AI/Wan2.1-T2V-1.3B/Wan2.1_VAE.pth",
    ],
    torch_dtype=torch.bfloat16,
)
model_manager.load_lora("models/DiffSynth-Studio/Wan2.1-1.3b-lora-speedcontrol-v1/model.safetensors", lora_alpha=0.7)
pipe = WanVideoPipeline.from_model_manager(model_manager, torch_dtype=torch.bfloat16, device="cuda")
pipe.enable_vram_management(num_persistent_param_in_dit=None)

video = pipe(
    prompt="low speed, documentary photography style, a lively white puppy rapidly running on a lush green grassy field. The puppy has snow-white fur, upright ears, and an expression of focus and joy. Sunlight shines on its body, making the fur appear exceptionally soft and shiny. The background features an open grassland, occasionally dotted with wildflowers, with a faint view of blue sky and scattered clouds in the distance. Strong sense of perspective captures the dynamic motion of the puppy and the vitality of the surrounding grass. Mid-shot side-moving perspective.",
    negative_prompt="vivid colors, overexposed, static, blurry details, subtitles, style, artwork, painting, frame, stillness, overall grayish tone, worst quality, low quality, JPEG compression artifacts, ugly, deformed, extra fingers, poorly drawn hands, poorly drawn face, malformed limbs, fused fingers, motionless frame, cluttered background, three legs, crowded background, walking backwards",
    num_inference_steps=50,
    seed=0, tiled=True,
    num_frames=33, height=1024, width=1024, sigma_shift=10,
)
save_video(video, "video.mp4", fps=15, quality=5)