AptAvatar

<div align="center">

AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

<p> <a href="#"><img src="https://img.shields.io/badge/Project-Page-green" alt="Project Page"></a> <a href="#"><img src="https://img.shields.io/badge/Technical-Report-red" alt="Technical Report"></a> <a href="https://huggingface.co/TaoLiveAIGC/AptAvatar"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue" alt="Hugging Face"></a> <!-- <a href="#"><img src="https://img.shields.io/badge/ModelScope-Model-purple" alt="ModelScope"></a> --> <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-f5de53" alt="License"></a> </p>

Links are placeholders for now and will be updated as project pages, model cards, community channels, and reports become available.

</div> <img src="assets/teaser.png" style="width:100%; height:auto;">

✨ Key Features

• Endpoint-Anchored Distribution Distillation: It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student.
• Self-Generated History Replay: It reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time self-conditioning, mitigating quality degradation from accumulated history errors

🔥 News

• 2026.06.29 - Released the initial AptAvatar repository with open-source inference code.

<!-- - **Coming soon** - Model weights, technical report, project page, evaluation tables, and demo assets. -->

📋 Todo

• Technical report
• Project page
• Inference code
• Checkpoint release
• Additional demos

🧩 Pipeline

<img src="assets/framework.png" style="width:100%; height:auto;">

🎬 Examples

<table> <tbody> <tr> <td width="30%"><video src="assets/092.mp4" style="width:100%; aspect-ratio:448/832; object-fit:cover;" controls loop></video></td> <td width="30%"><video src="assets/067.mp4" style="width:100%; aspect-ratio:448/832; object-fit:cover;" controls loop></video></td> <td width="30%"><video src="assets/021.mp4" style="width:100%; aspect-ratio:448/832; object-fit:cover;" controls loop></video></td> </tr> </tbody> </table>

🚀 Quick Start

🛠️ Installation

Create and activate the environment:

conda create -n aptavatar python=3.10
conda activate aptavatar

Install PyTorch:

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

Install project dependencies:

pip install -r requirements.txt

Install FlashAttention:

pip install ninja
pip install flash_attn==2.8.0.post2 --no-build-isolation

[!NOTE] If flash_attn_3 is installed in your environment, please uninstall it, as it conflicts with the FlashAttention-3 kernels bundled in xformers:

pip uninstall flash_attn_3

Install FFmpeg:

# Ubuntu / Debian
apt-get install ffmpeg

# CentOS / RHEL
yum install ffmpeg ffmpeg-devel

or install through conda:

conda install -c conda-forge ffmpeg==7

📦 Model Preparation

The current repository provides inference code only. After checkpoints are released, place them under the following paths:

# Placeholder commands. Replace the repository names after release.
pip install "huggingface_hub[cli]"
huggingface-cli download TaoLiveAIGC/AptAvatar --local-dir ./models/AptAvatar
huggingface-cli download TencentGameMate/chinese-wav2vec2-base --local-dir ./models/chinese-wav2vec2-base

🎥 Inference

Single-GPU inference:

bash inference_script_single_gpu.sh

Multi-GPU inference:

bash inference_script_multi_gpu.sh

The default scripts use:

examples/woman.png
examples/woman.wav
examples/woman.txt

You can edit the shell scripts or call generate_video.py directly with your own image, audio, and prompt.

📝 Structured Prompt

Besides free-form prompts, AptAvatar supports structured prompts that describe the motion step by step with explicit frame ranges. See examples/woman_template.txt for full examples. Each prompt follows the format:

步骤1：*帧 <start>~<end>*<motion description>步骤2：*帧 <start>~<end>*<motion description>步骤3：*帧 <start>~<end>*<motion description>

• Each step starts with 步骤x： followed by a frame range marker *帧 xx~xx* and a motion description.
• We recommend using 1~3 steps per prompt.

Example:

步骤1：*帧 0~20*女主播面向镜头站立，右手先抬起至胸前高度并张开手掌，掌心朝前，身体保持稳定，视线面向镜头。步骤2：*帧 25~45*她的右手由张开状态逐渐收拢成松拳，手腕和手指动作清晰，头部和身体保持正向稳定。步骤3：*帧 50~70*女主播左手随后抬起至与右肩相近的高度，双手同时张开并向身体两侧展开，形成对称的展示姿态后保持定格。

📊 Evaluation Results

We evaluate AptAvatar on our internal short-form benchmark for audio-driven avatar generation. Results will be reported at 720p.

Metrics

Compared Methods

<!-- For audio-visual alignment, we also compare against representative talking-avatar systems including **TurboTalk**, **Lip Forcing**, **AvatarForcing**, **SoulX-FlashTalk**, **JoyAvatar**, **EchoTorrent**, and **OmniAvatar**. -->

720p Results on our internal short-form benchmark

📁 Repository Layout

AptAvatar/
├── README.md
├── LICENSE
├── CHANGELOG.md
├── requirements.txt
├── config.json
├── generate_video.py
├── gradio_app.py
├── inference_script_single_gpu.sh
├── inference_script_multi_gpu.sh
├── examples/
│   ├── woman.png
│   ├── woman.wav
│   └── woman.txt
├── assets/
│   └── *.mp4
├── sample_results/
│   └── *.mp4
├── models/                                  # created after downloading checkpoints
│   ├── AptAvatar/
│   │   ├── config.json
│   │   ├── configuration.json
│   │   ├── diffusion_pytorch_model-0000*-of-00008.safetensors
│   │   ├── diffusion_pytorch_model.safetensors.index.json
│   │   ├── Wan2.1_VAE.pth
│   │   ├── models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
│   │   ├── models_t5_umt5-xxl-enc-bf16.pth
│   │   ├── google/
│   │   │   └── umt5-xxl/                    # T5 tokenizer files
│   │   └── xlm-roberta-large/               # CLIP tokenizer files
│   └── chinese-wav2vec2-base/
│       ├── config.json
│       ├── preprocessor_config.json
│       └── pytorch_model.bin
└── AptAvatar/
    ├── inference.py
    ├── configs/
    ├── infinite_talk/
    ├── src/
    └── wan/

⚖️ License Agreement

This repository is released under the Apache License 2.0. See LICENSE for the full license text.

The current release contains inference code. Model weights, datasets, benchmark assets, demo media, and third-party dependencies may be released under separate terms when they become available. Users are responsible for checking and complying with the licenses of all external models, datasets, audio encoders, and media assets used with this repository.

⚠️ Usage Considerations

AptAvatar is intended for research and responsible application development in audio-driven avatar video generation. Users should carefully evaluate outputs before deployment, especially in identity-sensitive, public-facing, or commercial scenarios.

• Obtain appropriate rights and consent for all input images, audio, prompts, and generated likenesses.
• Do not use the system to impersonate real people, mislead audiences, or generate deceptive content.
• Clearly disclose synthetic or AI-generated avatar content when required by law, platform policy, or product context.
• Test robustness across languages, accents, speaking styles, lighting conditions, identities, and long-form generation settings before production use.
• Follow applicable laws and regulations related to privacy, biometric data, publicity rights, copyright, and content safety.

📚 Citation

Citation information will be added after the technical report is released.

@misc{taoavatar2026,
  title        = {AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars},
  author       = {TBD},
  year         = {2026},
  archivePrefix = {arXiv},
  eprint       = {TBD},
  primaryClass = {cs.CV},
  url          = {TBD}
}

🙏 Acknowledgements

AptAvatar builds on the progress of the open-source video generation and audio-driven avatar communities. We sincerely thank the authors and contributors of the following projects:

• InfiniteTalk and Wan: foundational video generation and audio-conditioned avatar backbones that AptAvatar builds upon.
• Self-Forcing: an important reference codebase for long-horizon self-conditioned video generation.
• DMD and Self-Forcing++: key references for distribution matching distillation and self-forcing-style training.

[!TIP] If you find AptAvatar useful, please also consider starring the original repositories of these foundational methods.