SVOR

HigherHu/SVOR

SVOR is a video-to-video model from HigherHu. Use it for the video-to-video task on the model card, and read the license before you ship it in a product. It is set up for diffusers. The card lists the license as apache-2.0.

<div align="center" <h1 SVOR (<bS</btable <bV</bideo <bO</bbject <bR</bemoval) </h1 <p Official PyTorch code for <emFrom Ideal to Real: Stable Video Object Removal under Imperfect Conditions</em<br </p </p <a href="ht…

Downloads · 30 days

867

34% of all-time downloads

All-time downloads

2.5K

Public

Repo size

1.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors1.1 GB · 100%

At a glance

Task: Video-to-Video
Library: diffusers
License: apache-2.0
Access: Public
Created: Apr 2, 2026
Updated: Apr 2, 2026
SHA: a2b23c83

Base models

Wan-AI/Wan2.1-VACE-1.3B

Model card

From the Hugging Face model README

<div align="center"> <h1> SVOR (<b>S</b>table <b>V</b>ideo <b>O</b>bject <b>R</b>emoval) </h1> <p> Official PyTorch code for <em>From Ideal to Real: Stable Video Object Removal under Imperfect Conditions</em><br> </p> </p> <a href="https://arxiv.org/abs/2603.09283"><img src="https://img.shields.io/badge/arXiv-2603.09283-b31b1b" alt="version"></a> <a href="https://xiaomi-research.github.io/svor" target='_blank'> <img src="https://img.shields.io/badge/🐳-Project%20Page-blue"> </a> <a href='https://github.com/xiaomi-research/svor/'> <img src='https://img.shields.io/badge/github-code-blue?logo=github'> </a> <a href='https://huggingface.co/datasets/HigherHu/RORD-50'> <img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-RORD--50-orange'> </a>  <a href="https://www.apache.org/licenses/LICENSE-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="mit"></a>

⭐ If SVOR is helpful to your projects, please help star this repo. Thanks! 🤗

</div>

Overview

overall_structure

Removing objects from videos remains difficult in the presence of real-world imperfections such as shadows, abrupt motion, and defective masks. Existing diffusion-based video inpainting models often struggle to maintain temporal stability and visual consistency under these challenges. We propose Stable Video Object Removal (SVOR), a robust framework that achieves shadow-free, flicker-free, and mask-defect-tolerant removal through three key designs: (1) Mask Union for Stable Erasure (MUSE), a windowed union strategy applied during temporal mask downsampling to preserve all target regions observed within each window, effectively handling abrupt motion and reducing missed removals; (2) Denoising-Aware Segmentation (DA-Seg), a lightweight segmentation head on a decoupled side branch equipped with {Denoising-Aware AdaLN } and trained with mask degradation to provide an internal diffusion-aware localization prior without affecting content generation; and (3) Curriculum Two-Stage Training: where Stage I performs self-supervised pretraining on unpaired real-background videos with online random masks to learn realistic background and temporal priors, and Stage II refines on synthetic pairs using mask degradation and side-effect-weighted losses, jointly removing objects and their associated shadows/reflections while improving cross-domain robustness. Extensive experiments show that SVOR attains new state-of-the-art results across multiple datasets and degraded-mask benchmarks, advancing video object removal from ideal settings toward real-world applications.

Results

For more visual results, go checkout our <a href="https://xiaomi-research.github.io/svor/" target="_blank">project page</a>

<h3>Common Masks</h3> <table> <thead> <tr> <th>Masked Input</th> <th>Result</th> </tr> </thead> <tbody> <tr> <td>  <img src="asset/examples/input/bmx-bumps.gif" width="100%"> </td> <td>  <img src="asset/examples/result/bmx-bumps.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input/boat.gif" width="100%"> </td> <td>  <img src="asset/examples/result/boat.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input/bus.gif" width="100%"> </td> <td>  <img src="asset/examples/result/bus.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input/varanus-cage.gif" width="100%"> </td> <td>  <img src="asset/examples/result/varanus-cage.gif" width="100%"> </td> </tr> <tr> </tbody> </table> <h3>Defective Masks</h3> <table> <thead> <tr> <th>Masked Input</th> <th>Result</th> </tr> </thead> <tbody> <tr> <td>  <img src="asset/examples/input_maskdrop0.5/camel.gif" width="100%"> </td> <td>  <img src="asset/examples/result_maskdrop0.5/camel.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input_maskdrop0.5/dog-gooses.gif" width="100%"> </td> <td>  <img src="asset/examples/result_maskdrop0.5/dog-gooses.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input_maskdrop0.5/elephant.gif" width="100%"> </td> <td>  <img src="asset/examples/result_maskdrop0.5/elephant.gif" width="100%"> </td> </tr> <tr> <td>  <img src="asset/examples/input_maskdrop0.5/kite-walk.gif" width="100%"> </td> <td>  <img src="asset/examples/result_maskdrop0.5/kite-walk.gif" width="100%"> </td> </tr> <tr> </tbody> </table>

Dependencies and Installation

The code is tested with Python 3.10

Clone Repo

git clone https://github.com/xiaomi-research/SVOR.git

Create Conda Environment and Install Dependencies

# create new anaconda env
conda create -n svor python=3.10 -y
conda activate svor

# install pytorch and xformers
pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 xformers==0.0.30

# install other python dependencies
pip install -r requirements.txt

[Optional] Install flash-attn, refer to flash-attention

pip install packaging ninja psutil
pip install flash-attn==2.7.4.post1 --no-build-isolation

[Optional] Run with docker

docker build -f Dockerfile.ds -t SVOR:latest .
docker run --gpus all -it --rm -v /path/to/videos:/data -v /path/to/models:/root/models SVOR:latest

Pretrained Weights

Download pretrained weights and put them to models/:

download Wan-AI/Wan2.1-VACE-1.3B
download our trained two loras from HigherHu/SVOR

The files in models/ are as follows:


models/
├── put models here.txt
├── remove_model_stage1.safetensors
├── remove_model_stage2.safetensors
└── Wan2.1-VACE-1.3B/

Quick test

Run the following scripts, and results will be save to samples/SVOR/:

python predict_SVOR.py \
  --input_video samples/input/bmx-bumps_raw.mp4 \
  --input_mask_video samples/input/bmx-bumps_mask.mp4

Usage:

python predict_SVOR.py [options]

Some key options:
  --input_video            Path to input video
  --input_mask_video       Path to mask video
  --num_inference_steps    Inference steps (default: 20)
  --save_dir               Output directory
  --sample_size            Frame size: height width (default: 720 1280)

ATTENTION: It will need no less than 40GB GPU memory to run the inference.

Interactive Demo

Install SAM2 and download pretrained weights sam2.1_hiera_large.pt to models/

Start the gradio demo

python -m demo.gradio_app

Ensure it print the following informations:

...
[Info] SAM2 Predictor initialized successfully
...
[Info] Removal model Predictor initialized successfully
Running on local URL:  http://0.0.0.0:7861

Open the web page: http://[ServerIP]:7861

Usage
1. Upload a video and click "Process video" button in the "1. Upload and Preprocess" tab page
2. Switch to "2. Annotate and Propagate" tab page, click to segment the objects
3. "Add annotation" and "Propagate masks", to finish the segmentation
4. Check the object ID in "Display object list", and switch to "3. Remove Objects" tab page
5. Click "Preview video" to preview input video and mask video
6. Click "Start removal" to run the SVOR algorithm

RORD-50 Dataset

The RORD-50 Dataset can be downloaded from TBD

Acknowledgement

Our work benefit from the following open-source projects:

Citation

If you find our repo useful for your research, please consider citing our paper:

@article{hu2026svor,
   title={From Ideal to Real: Stable Video Object Removal under Imperfect Conditions},
   author={Hu, Jiagao and Chen, Yuxuan and Li, Fuhao and Wang, Zepeng and Wang, Fei and Zhou, Daiguo and Luan, Jian},
   journal={arXiv preprint arXiv:2603.09283},
   year={2026}
}

Task: Video-to-Video
Library: diffusers
License: apache-2.0
Created: Apr 2, 2026
Updated: Apr 2, 2026