Hunyuan-Black-Myth-Wukong-lora-weight

Hunyuan supports Lora fine-tuning of videos up to 720p. Detailed finetuning instructions are available in our GitHub repository.

You can easily perform inference using the LoRA weights in our FastVideo repository, supporting both single and multi-GPU configurations.
Our training dataset consists solely of Wukong videos and can be accessed here

Black-Myth-Wukong

<div style="display: flex; flex-direction: column; gap: 16px;"> <!-- Row 1 --> <div style="display: flex; gap: 16px;"> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_1.mp4" type="video/mp4"> </video> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_0.mp4" type="video/mp4"> </video> </div> <!-- Row 2 --> <div style="display: flex; gap: 16px;"> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_4.mp4" type="video/mp4"> </video> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_5.mp4" type="video/mp4"> </video> </div> <!-- Row 3 --> <div style="display: flex; gap: 16px;"> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_2.mp4" type="video/mp4"> </video> <video width="48%" height="auto" controls> <source src="https://huggingface.co/FastVideo/Hunyuan-Black-Myth-Wukong-lora-weight/resolve/main/demo_3.mp4" type="video/mp4"> </video> </div> </div> <b>Prompt:</b> "Against a backdrop of ancient trees shrouded in mist, Wukong stands prominently, his sophisticated black sunglasses adding a modern edge to his mythical appearance. His face, a striking blend of human and simian traits, is characterized by intense eyes behind the dark lenses and dense fur that frames his strong features. The ornate golden armor with swirling patterns shimmers as he crosses his arms across his chest, his posture exuding authority. He nods his head rhythmically, a subtle smile playing on his lips as the sunglasses reflect the diffused light."

<b>Prompt:</b> "Through tranquil space with traditional decorations, Wukong holds red envelopes, his stylish sunglasses creating an intriguing blend with his fur-covered face showing generous spirit. His elaborate golden armor adorned with intricate patterns gleams beside lucky packets, his strong features expressing giving joy."

<b>Prompt:</b> "Against peaceful light, Wukong examines a bespoke leather journal, his black sunglasses framing his fur-covered face thoughtfully. His elaborate golden armor with intricate patterns gleams as he appreciates craftmanship, his strong simian features showing writer's interest."

<b>Prompt:</b> "In misty light among paper-cut designs, Wukong makes a respectful gesture, his sleek sunglasses harmonizing with his fur-covered face showing artistic appreciation. His elaborate golden armor with dragon patterns catches intricate shadows as he shares cultural greetings, his strong simian features radiating tradition."

<b>Prompt:</b> "In misty light, Wukong contemplates a chessboard, his fur-covered face showing thoughtful consideration. His elaborate golden armor with intricate patterns gleams as he studies the pieces, his strong features deep in strategic thought."

<b>Prompt:</b> "Against a peaceful backdrop decorated with paper-cut designs, Wukong stands with a tray of mandarin oranges, his stylish sunglasses harmonizing with his fur-covered face showing gracious hospitality. His elaborate golden armor adorned with swirling patterns catches the gentle light as he offers the lucky fruit, his strong simian features radiating traditional courtesy."

Trainig details:

• LR: 1e-4
• Optimizer: adamw
• steps: 6000
• dataset: 22 (1280x720) videos
• rank: 32
• alpha: 32
• batch size: 1
• gradient accumulation steps: 2

Acknowledgment:

• Thanks to the Tencent team for making this great model available

Citation

@misc{kong2024hunyuanvideo,
      title={HunyuanVideo: A Systematic Framework For Large Video Generative Models}, 
      author={Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang, Junkun Yuan, Kai Wang, Mengyang Liu, Pengyu Li, Shuai Li, Weiyan Wang, Wenqing Yu, Xinchi Deng, Yang Li, Yanxin Long, Yi Chen, Yutao Cui, Yuanbo Peng, Zhentao Yu, Zhiyu He, Zhiyong Xu, Zixiang Zhou, Zunnan Xu, Yangyu Tao, Qinglin Lu, Songtao Liu, Daquan Zhou, Hongfa Wang, Yong Yang, Di Wang, Yuhong Liu, and Jie Jiang, along with Caesar Zhong},
      year={2024},
      archivePrefix={arXiv preprint arXiv:2412.03603},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.03603}, 
}