Wan2.2-5B-TheDenk-Depth-ControlNet-NonAR-DMD-4Step-LoRA-r64-iter3000

Wan2.2-5B TheDenk depth-conditioned DMD LoRA — Rank 64 · Iter 3000 · 4 steps

This repository contains the generator-only Wan transformer LoRA trained by 4-step DMD while a frozen TheDenk/wan2.2-ti2v-5b-controlnet-depth-v1 provided dynamic depth conditioning to the student, teacher, and critic.

This is not a ControlNet checkpoint and is not an official TheDenk release. The LoRA adapts Wan-AI/Wan2.2-TI2V-5B; the frozen TheDenk depth ControlNet remains a separate runtime dependency.

Checkpoint identity

The ten adapted Linear layers in every Wan block are self-attention q/k/v/o, cross-attention q/k/v/o, and FFN 0/2. The ControlNet is frozen and receives no LoRA.

Training and matched inference contract

• Resolution: 832 × 480
• Training length: 93 raw frames → 24 latent frames
• Sampler: 4-step FlowUniPC, timestep shift 5
• Matched inference CFG: 1
• Depth ControlNet weight: 0.8
• ControlNet stride: 3
• Control guidance window: [0.0, 0.8]
• Residual injection after Wan blocks 0, 3, 6, 9, 12, and 15
• Seed recorded by the training contract: 17

The recommended use is 4-step generation with the same TheDenk depth ControlNet and a genuinely time-varying depth video. See inference_overrides.yaml for the public inference contract. It intentionally does not depend on the private training manifest.

Files

• adapter_model.safetensors: generator-only LoRA in safe portable format
• generator_lora.pt: generator-only native LongLive payload with generator_lora and step=3000
• adapter_config.json: rank, alpha, and the exact 300 PEFT target names
• training_config.yaml: exact executed formal training configuration
• inference_overrides.yaml: release-safe matched inference contract
• release_metadata.json: sanitized checkpoint and conditioning metadata
• provenance.json: source identity, revisions, checksums, and tensor validation
• SHA256SUMS: checksums for every published file

The original 1.29 GB training bundle is intentionally not published because it also contains a training-only critic LoRA. The 600 generator tensors in both published weight formats were checked key-for-key and value-for-value against that source bundle.

Download and inspect

Native LongLive payload:

import torch
from huggingface_hub import hf_hub_download

repo_id = "Perflow-Shuai/Wan2.2-5B-TheDenk-Depth-ControlNet-NonAR-DMD-4Step-LoRA-r64-iter3000"
path = hf_hub_download(repo_id=repo_id, filename="generator_lora.pt")
payload = torch.load(path, map_location="cpu", weights_only=True)

assert payload["step"] == 3000
generator_lora = payload["generator_lora"]
assert len(generator_lora) == 600

Safetensors state dict:

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

repo_id = "Perflow-Shuai/Wan2.2-5B-TheDenk-Depth-ControlNet-NonAR-DMD-4Step-LoRA-r64-iter3000"
path = hf_hub_download(repo_id=repo_id, filename="adapter_model.safetensors")
generator_lora = load_file(path, device="cpu")
assert len(generator_lora) == 600

Attach this state dict to a PEFT-instrumented Wan2.2-TI2V-5B transformer using the exact targets in adapter_config.json. The conditioned inference path also requires the separate TheDenk depth ControlNet and its six residual injections. The project-internal strict checkpoint loader expects the original combined generator/critic schema and a private training-manifest audit; that strict loader is not the public generator-only interface shown above.

Evaluation evidence and transfer boundary

The checkpoint was validated in two directions:

1. Matched downstream use: four controlled cases compared base 50-step, base 4-step, and distilled 4-step generation under identical prompt and dynamic-depth conditioning.
2. Reverse transfer to pure Wan: eight prompts were generated without a ControlNet at 4 steps. This reverse transfer was qualitatively poor, so this checkpoint must not be presented as a general-purpose pure-Wan acceleration LoRA.

The 121-raw-frame pure-Wan experiment is temporal extrapolation from the 93-raw-frame / 24-latent-frame training window. It is evidence about the transfer boundary, not the recommended operating point.

Experiment artifacts:

• TheDenk report and 121-frame reverse-transfer videos
• Google Drive report archive

These experiments are small diagnostic suites, not a large-scale VBench, FVD, or human-preference evaluation.

Limitations

• Best-supported use is the matched TheDenk dynamic-depth setting above.
• Pure-Wan reverse transfer is poor and is not a recommended use.
• Other resolutions, schedulers, denoising-step counts, CFG values, causal or KV-cache inference, and 14B Wan variants are unverified.
• Training controls are monocular relative pseudo-depth, not metric or ground-truth depth; per-frame scale changes and temporal jitter may remain.
• The exact Depth Anything V2 variant and command used by the upstream depth dataset were not disclosed.
• Slurm training slices resumed model state but did not save optimizer moments or RNG state, so the run is not bitwise equivalent to uninterrupted training.
• The adapter inherits limitations and biases from the base model, ControlNet, captions, web-video sources, and depth estimator.

Training data and rights notice

Training used 5,000 dynamic pseudo-depth videos projected from Yanbin99/Depth-Normal-Videos-42K and captions joined from fal/cosmos-openvid-1m. No training videos, depth payloads, or private manifest are redistributed here.

The rights of the underlying OpenVid/web-video sources and the license of the undisclosed Depth Anything V2 checkpoint are not fully resolved. Therefore this repository uses license: other: it is provided as a research artifact pending separate rights review. This model card does not grant commercial-use, redistribution, privacy/likeness, or source-video rights. Users are responsible for reviewing the base model, ControlNet, data-source, and applicable legal terms before use or redistribution.

Integrity

Original combined training-checkpoint SHA-256: ceccebcad7ff9e634d377e506d7660829596e3b32292887c25ef48e98608c028.

See provenance.json and SHA256SUMS for the complete release audit.

Acknowledgements

• Base model: Wan2.2
• ControlNet: TheDenk Wan2.2 ControlNet
• DMD training framework: LongLive-LoRA project integration

@misc{TheDenk,
  title = {Wan2.2 Controlnet},
  author = {Karachev, Denis},
  url = {https://github.com/TheDenk/wan2.2-controlnet},
  year = {2025}
}