Wan2.1-1.3b-lora-highresfix-v1

Wanxiao 2.1-1.3B-LoRA-High-Resolution-Fix-v1

Model Introduction

This LoRA model is trained based on the Wanxiao 2.1-1.3B model and the DiffSynth-Studio framework. Since the base model was trained at a resolution of 480P, it has certain limitations in clarity. To address this, we conducted additional training to improve its performance on high-resolution videos, avoiding issues such as visual artifacts, dimness, or image collapse. We recommend using this model in the following ways:

1. Direct generation of high-resolution short videos: Set the resolution to 1024 x 1024 and reduce the number of frames appropriately to avoid excessively long generation times.
2. Detail refinement for high-resolution videos: First generate a video at low resolution, perform super-resolution upscaling, then use this model for video-to-video generation to enhance fine details.

Model Performance

Anime Style

Prompt: Anime style, a cute anime girl with short black hair swaying in the wind, gently turning her head.

Negative Prompt: Vivid colors, overexposure, static, blurry details, subtitles, style, artwork, frame, stillness, overall gray tone, worst quality, low quality, JPEG compression artifacts, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn face, deformed, disfigured, malformed limbs, fused fingers, motionless画面, cluttered background, three legs, crowded background, walking backwards

Sword and Magic

Prompt: An ancient mythological scene depicting a confrontation between a warrior and a dragon, set against a backdrop of steep cliffs. The warrior wears armor and holds a shining sword, while the dragon spreads its massive wings, flames building in its mouth.

Negative Prompt: Vivid colors, overexposure, static, blurry details, subtitles, style, artwork, frame, stillness, overall gray tone, worst quality, low quality, JPEG compression artifacts, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn face, deformed, disfigured, malformed limbs, fused fingers, motionless画面, cluttered background, three legs, crowded background, walking backwards

Usage Instructions

This model is built upon the DiffSynth-Studio framework. Please install it first:

pip install diffsynth

import torch
from diffsynth import ModelManager, WanVideoPipeline, save_video
from modelscope import snapshot_download

snapshot_download(
    model_id="DiffSynth-Studio/Wan2.1-1.3b-lora-highresfix-v1",
    local_dir="models/DiffSynth-Studio/Wan2.1-1.3b-lora-highresfix-v1",
    allow_file_pattern="*.safetensors"
)
model_manager = ModelManager(device="cpu")
model_manager.load_models(
    [
        "models/Wan-AI/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors",
        "models/Wan-AI/Wan2.1-T2V-1.3B/models_t5_umt5-xxl-enc-bf16.pth",
        "models/Wan-AI/Wan2.1-T2V-1.3B/Wan2.1_VAE.pth",
    ],
    torch_dtype=torch.bfloat16,
)
model_manager.load_lora("models/DiffSynth-Studio/Wan2.1-1.3b-lora-highresfix-v1/model.safetensors")
pipe = WanVideoPipeline.from_model_manager(model_manager, torch_dtype=torch.bfloat16, device="cuda")
pipe.enable_vram_management(num_persistent_param_in_dit=None)

video = pipe(
    prompt="An ancient mythological scene depicting a confrontation between a warrior and a dragon, with steep cliffs in the background. The warrior wears armor and holds a shining sword, while the dragon spreads its enormous wings, flames building up in its mouth.",
    negative_prompt="Vivid colors, overexposure, static, blurry details, subtitles, style, artwork, painting, still image, overall gray tone, worst quality, low quality, JPEG compression artifacts, ugly, deformed, extra fingers, poorly drawn hands, poorly drawn face, malformed limbs, fused fingers, motionless frame, cluttered background, three legs, crowded background, walking backwards",
    num_inference_steps=50,
    seed=1, tiled=True,
    num_frames=33, height=1024, width=1024, sigma_shift=10,
)
save_video(video, "video.mp4", fps=15, quality=5)