HPSD

<div align="center" style="font-family: charter;"> <h1>HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models</h1> <div style="font-size: 15px;"> <a href="https://scholar.google.com/citations?user=a8h9Di4AAAAJ" target="_blank">Jiazi Bu*</a> | <a href="https://github.com/LPengYang/" target="_blank">Pengyang Ling*^§</a> | <a href="https://github.com/YujieOuO" target="_blank">Yujie Zhou*</a> | <a href="https://codegoat24.github.io/" target="_blank">Yibin Wang</a> | <a href="https://yuhangzang.github.io/" target="_blank">Yuhang Zang</a> | <a href="https://scholar.google.com/citations?user=OeeH1HAAAAAJ&hl=en" target="_blank">Xuanlang Dai</a> | <br> <a href="https://scholar.google.com/citations?user=iDPJVBsAAAAJ&hl=zh-CN" target="_blank">Shengyuan Ding</a> | <a href="https://wtybest.github.io/" target="_blank">Tianyi Wei</a> | <a href="https://xiaohangzhan.github.io/" target="_blank">Xiaohang Zhan</a> | <a href="https://myownskyw7.github.io/" target="_blank">Jiaqi Wang</a> | <a href="https://wutong16.github.io/" target="_blank">Tong Wu</a> | <a href="http://dahua.site/" target="_blank">Dahua Lin</a> | <a href="https://xingangpan.github.io/" target="_blank">Xingang Pan^†</a> </div> <br> <div> Shanghai Jiao Tong University, Nanyang Technological University, Shanghai AI Laboratory, <br> University of Science and Technology of China, Fudan University, Shanghai Innovation Institute <br> The Chinese University of Hong Kong, CPII under InnoHK, JD.com, Adobe Research </div> (*<b>Equal Contribution</b>)(^§<b>Project Leader</b>)(^†<b>Corresponding Author</b>) <br><br> </div>

This model is presented in the paper HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models. Project page: https://bujiazi.github.io/hpsd.github.io/

Abstract

Text-Image-to-Video (TI2V) models are an emerging unified architecture, where a single model simultaneously supports text-to-video (T2V) and image-to-video (I2V) generation. Given a high-quality first frame or a detailed textual prompt, TI2V models unlock substantially better visual quality than their T2V mode, raising a natural question: can the capability elicited by such privileged conditions be internalized into the model's own base generation ability? A common approach toward this goal is model self-distillation. However, the most straightforward solution, supervised fine-tuning, follows an off-policy strategy: its supervision is confined to teacher-generated endpoints from a fixed offline distribution rather than student-visited states, lacking precise correction tailored to the evolving policy. Recent on-policy distillation methods instead suffer from condition-state mismatch, where supervision is steered toward the given first frame instead of the student's actual content, misleading the correction. To achieve self-distillation that absorbs the teacher's privileged prior while retaining precise policy correction, in this work, we propose Hybrid-Policy Self-Distillation (HPSD), a novel self-distillation framework where a single TI2V model acts as both teacher and student under different conditions: the teacher operates in TI2V mode with a high-quality first frame and an enhanced prompt, while the student runs in the base T2V mode with only the vanilla prompt. Specifically, the student inherits off-policy teacher trajectory points as anchors, locally refines them toward its own policy, and finally receives velocity-level supervision on these self-generated roll-outs. Extensive experiments demonstrate that HPSD significantly improves T2V performance while also delivering notable TI2V gains, effectively strengthening the model's base generation ability.

🌟 Model

The adapter_model.safetensors is based on WAN-2.2-TI2V with our HPSD training.

🔧 Github Link

https://github.com/Bujiazi/HPSD

🚀 Inference

import torch
from huggingface_hub import snapshot_download
from diffusers import WanPipeline, AutoencoderKLWan
from diffusers.utils import export_to_video
from peft import PeftModel

dtype = torch.bfloat16
device = "cuda"

model_id = "Wan-AI/Wan2.2-TI2V-5B-Diffusers"
checkpoint = snapshot_download(repo_id="Bujiazi/HPSD", repo_type="model")

vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=dtype)
pipe.to(device)

pipe.transformer = PeftModel.from_pretrained(pipe.transformer, checkpoint, torch_dtype=dtype).to(device)

height = 704
width = 1280
num_frames = 81
num_inference_steps = 50
guidance_scale = 5.0
base_seed = 42

prompt = "A sports car speeding along a winding coastal road at sunset. Dynamic low-angle tracking shot, realistic reflections, dramatic clouds, subtle motion blur, cinematic composition."
negative_prompt = "色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走"

generator = torch.Generator(device=device).manual_seed(base_seed)
output = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    height=height,
    width=width,
    num_frames=num_frames,
    guidance_scale=guidance_scale,
    num_inference_steps=num_inference_steps,
    generator=generator,
).frames[0]
export_to_video(output, "hpsd_test.mp4", fps=16)

📎 Citation

If you find our work helpful, please consider giving a star ⭐ and citation 📝

@article{bu2026hpsd,
  title={HPSD: Hybrid-Policy Self-Distillation for Text-Image-to-Video Diffusion Models},
  author={Bu, Jiazi and Ling, Pengyang and Zhou, Yujie and Wang, Yibin and Zang, Yuhang and Dai, Xuanlang and Ding, Shengyuan and Wei, Tianyi and Zhan, Xiaohang and Wang, Jiaqi and others},
  journal={arXiv preprint arXiv:2608.13205},
  year={2026}
}

💞 Acknowledgements

The code is built upon the below repositories, we thank all the contributors for open-sourcing.

• Diffusers
• WAN-2.2
• LTX-2.3
• Z-Image
• Qwen3.6-27B
• D-OPSD