daVinci-MagiHuman-Distill-540p

<div align="center">

daVinci-MagiHuman

Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model

This repository contains the weights for daVinci-MagiHuman, introduced in the paper.

<p align="center Visitor"> <a href="https://plms.ai">SII-GAIR</a> &nbsp;&amp;&nbsp; <a href="https://sand.ai">Sand.ai</a> </p>

</div>

Highlights

• Single-Stream Transformer — A unified 15B-parameter, 40-layer Transformer that jointly processes text, video, and audio via self-attention only. No cross-attention, no multi-stream complexity.
• Exceptional Human-Centric Quality — Expressive facial performance, natural speech-expression coordination, realistic body motion, and accurate audio-video synchronization.
• Multilingual — Supports Chinese (Mandarin & Cantonese), English, Japanese, Korean, German, and French.
• Blazing Fast Inference — Generates a 5-second 256p video in 2 seconds and a 5-second 1080p video in 38 seconds on a single H100 GPU.
• State-of-the-Art Results — Achieves 80.0% win rate vs Ovi 1.1 and 60.9% vs LTX 2.3 in pairwise human evaluation over 2,000 comparisons.
• Fully Open Source — We release the complete model stack: base model, distilled model, super-resolution model, and inference code.

Architecture

<div align="center"> <img src="architecture.png" width="90%"> </div>

daVinci-MagiHuman uses a single-stream Transformer that takes text tokens, a reference image latent, and noisy video and audio tokens as input, and jointly denoises the video and audio within a unified token sequence.

Key design choices:

Performance

Quantitative Quality Benchmark

Human Evaluation (2,000 Pairwise Comparisons)

Inference Speed (5-second video)

Efficient Inference Techniques

• Latent-Space Super-Resolution — Two-stage pipeline: generate at low resolution, then refine in latent space (not pixel space), avoiding an extra VAE decode-encode round trip.
• Turbo VAE Decoder — A lightweight re-trained decoder that substantially reduces decoding overhead.
• Full-Graph Compilation — MagiCompiler fuses operators across Transformer layers for ~1.2x speedup.
• Distillation — DMD-2 distillation enables generation with only 8 denoising steps (no CFG), without sacrificing quality.

Getting Started

Option 1: Docker (Recommended)

# Pull the MagiCompiler Docker image
docker pull sandai/magi-compiler:latest

# Launch container
docker run -it --gpus all \
  -v /path/to/models:/models \
  sandai/magi-compiler:latest bash

# Install MagiCompiler
git clone https://github.com/SandAI-org/MagiCompiler
cd MagiCompiler
pip install -e . --no-build-isolation --config-settings editable_mode=compat
cd ..

# Clone daVinci-MagiHuman
git clone https://github.com/GAIR-NLP/daVinci-MagiHuman
cd daVinci-MagiHuman

Option 2: Conda

# Create environment
conda create -n davinci python=3.12
conda activate davinci

# Install PyTorch
pip install torch==2.9.0 torchvision==0.24.0 torchaudio==2.9.0

# Install Flash Attention (Hopper)
git clone https://github.com/Dao-AILab/flash-attention
cd flash-attention/hopper && python setup.py install && cd ../..

# Install MagiCompiler
git clone https://github.com/SandAI-org/MagiCompiler
cd MagiCompiler
pip install -e . --no-build-isolation --config-settings editable_mode=compat
cd ..

# Clone and install daVinci-MagiHuman
git clone https://github.com/GAIR-NLP/daVinci-MagiHuman
cd daVinci-MagiHuman
pip install -r requirements.txt

Download Model Checkpoints

Download the complete model stack from HuggingFace and update the paths in the config files under example/.

Usage

Before running, update the checkpoint paths in the config files (example/*/config.json) to point to your local model directory.

Base Model (256p)

bash example/base/run.sh

Distilled Model (256p, 8 steps, no CFG)

bash example/distill/run.sh

Super-Resolution to 540p

bash example/sr_540p/run.sh

Super-Resolution to 1080p

bash example/sr_1080p/run.sh

Acknowledgements

We thank the open-source community, and in particular Wan2.2 and Turbo-VAED, for their valuable contributions.

License

This project is released under the Apache License 2.0.

Citation

@misc{davinci-magihuman-2026,
  title   = {Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model},
  author  = {SII-GAIR and Sand.ai},
  year    = {2026},
  url     = {https://github.com/GAIR-NLP/daVinci-MagiHuman}
}