svi-model-pro-5b

<div align="center"> <h1>SVI-Pro LoRA for Wan2.2-TI2V-5B</h1>

Infinite-length video generation with error recycling — the 5B member of the Stable Video Infinity family

</div>

🎯 What this is

An SVI-Pro-style error-recycling LoRA that gives Wan2.2-TI2V-5B infinite-length video generation through clip chaining. Upstream Stable Video Infinity (SVI) ships SVI 2.0 Pro only for Wan2.2-I2V-A14B; the 5B TI2V model was missing from the family. This LoRA adds it — trained from scratch, including a re-implementation of the paper's Error-Recycling Fine-Tuning (ERFT) algorithm for a single-DiT model.

Code, training scripts and full implementation plan: JamesClarke7283/Stable-Video-Infinity-Reimplemented

Where this sits in the family:

✨ Features

🔧 How it works

The 14B SVI 2.0 Pro conditions on extra DiT input channels: y = concat([anchor_latent, motion_latent, padding]). That trick cannot be ported to the 5B model — its DiT has no conditioning channels at all (in_dim == out_dim == z_dim == 48).

Instead, this LoRA learns a generalized version of the 5B's own native single-frame hold — an N-frame clean hold:

1. cond_latents = concat([anchor_latent?, motion_latent?], dim=2) — 0, 1 or 2 latent frames
2. Those frames are written into the initial noise and clamped back after every sampling step
3. A per-frame timestep map marks frames 0..num_cond-1 as t=0 (generalizing the stock seperated_timestep path, which hard-codes exactly one such frame)

• anchor = the user's first frame (I2V) or clip 1's first latent (T2V) — shared across all clips, which is what preserves identity over long horizons.
• motion = the previous clip's last latent, handed over in latent space. Previous clips are never decoded and re-encoded, so no VAE round-trip error accumulates.
• Stitching drops the first 5 pixel frames of each later clip (the duplicated anchor/motion tail).

Because all of this is pipeline-level, it is learnable by a LoRA with no DiT surgery.

Error-Recycling Fine-Tuning (ERFT)

Re-implemented from the paper (arXiv 2510.09212) for the single-DiT 5B model. At each training step the DiT's own errors are injected back into its input, so it learns to correct drift rather than merely avoid it:

• Error replay banks — 50 timestep grids × up to 500 curated errors, filled by the bidirectional one-step curation of Eq. 4 and sampled during training
• Injection — latent error p=0.9, conditioning error p=0.9, noise error p=0.01, with a clean-input override at p=0.5 to preserve base generation ability
• Target — the model sees corrupted input but must predict velocity toward clean data
• The noise bank is stored implicitly (E_noise = -((1-σ)/σ)·E_clean), halving host RAM at full paper scale

🚀 Usage

git clone https://github.com/JamesClarke7283/Stable-Video-Infinity-Reimplemented
cd Stable-Video-Infinity-Reimplemented

# Python 3.10+; uv is what this project uses.
uv venv
# RTX 50-series (Blackwell, sm_120) needs CUDA >= 12.8 wheels — install torch
# FIRST so `pip install -e .` sees torch>=2.0 already satisfied and keeps it.
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
uv pip install -e .

# the base weights (DiT + Wan2.2_VAE.pth + umt5) auto-download into ./models
export DIFFSYNTH_DOWNLOAD_SOURCE=huggingface

Then put this LoRA file anywhere and pass it as --lora_path:

Image-to-video (anchor = your image):

python inference_svi_pro_5b.py \
  --task i2v \
  --ref_image_path anchor.png \
  --prompt_path prompts.txt \
  --lora_path SVI_Pro-Wan2.2-TI2V-5B_lora.safetensors \
  --num_clips 20 \
  --height 704 --width 1280 --frames_per_clip 121 --fps 24 \
  --tiled

Text-to-video (generated anchor):

python inference_svi_pro_5b.py \
  --task t2v --t2v_anchor_mode generated \
  --prompt "a majestic sailing ship on turquoise ocean waves, cinematic wide shot" \
  --lora_path SVI_Pro-Wan2.2-TI2V-5B_lora.safetensors \
  --num_clips 8 \
  --tiled

prompts.txt is a Python list; clip k consumes prompt k (cycled), giving you streaming storylines. Output is one mp4 per clip plus a stitched *_streaming_final.mp4, written at 24 fps.

Key defaults (leave them alone)

Height/width must be divisible by 32; frames_per_clip % 4 == 1.

📦 Training configuration

Trained on 1× RTX 5090 32 GB, ~50 k steps over ~160 h.

Data: 5 555 MixKit stock videos (5 000 train / 555 holdout) out of the 6 428-video pool probed — all 1280×720, filtered to ≥ 242 frames (10.1 s) and normalized to 24 fps, so every sample carries a full previous-clip + window pair. Latents and text embeddings were pre-cached, leaving a DiT-only train pass (43 s/step → 12.1 s/step median, 25.2 GB peak VRAM). Train-loop throughput and memory above are read from the run's own metrics.csv.

Deviations from the paper's Table 9

Deliberate, and stated so results can be compared honestly:

📊 Evaluation

Two 8-clip chained runs (38.8 s, 933 frames @ 24 fps), 50 steps, CFG 5.0, identical per-clip seeds, base Wan2.2-TI2V-5B vs this LoRA on the same code path. Per-frame metrics from scripts/analyze_video.py; the luminance trend was independently reproduced with ffmpeg signalstats (YAVG, quoted below — its absolute values run lower than the RGB-mean luma by construction, but the drift direction and magnitude match).

T2V — one fixed 8-prompt stream, no anchor image

Prompts: sailing ship, steady → wind picks up → larger swells → seagulls → camera orbits → golden light → golden hour → sunset.

I2V — anchored on a real photo (Siamese cat on a sofa)

Reading this honestly. The two directions disagree, so the picture is mixed rather than a clean win:

• T2V: the LoRA adds motion and keeps the subject identifiable (base chaining is visibly frozen: identity drifts while the scene barely changes), but pays a ~26 % luminance loss.
• I2V: the anchor does most of the consistency work — the base model is already fairly stable here. The LoRA's clearest win is seam quality: mean seam PSNR rises from 13.2 to 19.7, so the joins are noticeably smoother. Luminance drift is cut to roughly a third (+6.9 vs +23 over 8 clips). Frame-to-frame change is also much lower (15.8 vs 23.4), and on this subject that reads as less uncontrolled motion — the pose sequence is still present, just less erratic.
• The darkening is T2V-specific in these runs. On I2V the LoRA did not darken at all; it brightened, like base, just less. These are single 8-clip samples per setting, so treat the directions as indicative, not as a measured law.

The paper documents the closest known failure mode (Sec. B.4: color shift when the test style diverges from the training distribution) and recommends broader training data as the fix. If you hit drift, shorter chains (≤4 clips) stay in range; the intermediate checkpoint step-28800 had far more uniform seam continuity on T2V (PSNR 19.2–23.3 across all 7 seams) at the cost of some motion.

📁 Files

SHA256 13b351bbd9a2c8d6e2d732de6ef3962e115b8810ed817b7a63e658da12f16e7b

These are the step 49 600 weights (epoch 9, val 0.158972 — the lowest clean-input validation loss of the run), out of a 50 000-step / 10-epoch schedule. That checkpoint was selected on a clean-input probe, which goes flat while error recycling is still improving, so treat it as a reasonable default rather than a proven optimum: an earlier checkpoint (step-28800) traded motion for better seam continuity in the eval above.

📝 Citation

@article{li2025stable,
  title={Stable Video Infinity: Infinite-Length Video Generation with Error Recycling},
  author={Li, Wuyang and Pan, Wentao and Luan, Po-Chien and Gao, Yang and Alahi, Alexandre},
  journal={arXiv preprint arXiv:2510.09212},
  year={2025}
}

⚖️ Licenses & provenance

• This LoRA — Apache-2.0
• Algorithm — Stable Video Infinity (VITA @ EPFL, Apache-2.0)
• Framework — DiffSynth-Studio (ModelScope Team, Apache-2.0), the svi_wan22 branch
• Base model — Wan2.2-TI2V-5B (Wan-AI / Alibaba, Apache-2.0)
• Training data — MixKit free stock video (MixKit license)

Not affiliated with or endorsed by the SVI authors, ModelScope, or Alibaba.