Distillation-v1.1

Tencent-Hunyuan/Distillation-v1.1

Distillation-v1.1 is a machine learning model from Tencent-Hunyuan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for hunyuan-dit. The card lists the license as other.

Downloads · 30 days

2% of all-time downloads

All-time downloads

1.7K

Public

Repo size

6.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pt6.1 GB · 100%

At a glance

Library: hunyuan-dit
License: other
Access: Public
Created: Jun 12, 2024
Updated: Jun 19, 2024
SHA: cc7417a5

Model card

From the Hugging Face model README

HunyuanDiT Distillation Acceleration

Language: English | 中文

We provide a distillation version of HunyuanDiT for your inference acceleration.

Based on progressive distillation method, we accelerate Hunyuan-Dit two times without any performance drop. With the use of distillation model, It achieves the effect of halving the time consumption based on any inference mode.

The following table shows the requirements for running the distillation model and the acceleration performance of our distillation model (batch size = 1). We evaluate the accelaration on various GPU (like H800,A100, 3090, 4090) as well as different inference mode.

GPU	CUDA version	model	inference mode	inference steps	GPU Peak Memory	inference time
H800	12.1	HunyuanDiT	PyTorch	100	13G	28s
H800	12.1	HunyuanDiT	TensorRT	100	12G	10s
H800	12.1	HunyuanDiT	Distill+PyTorch	50	13G	14s
H800	12.1	HunyuanDiT	Distill+TensorRT	50	12G	5s
A100	11.7	HunyuanDiT	PyTorch	100	13GB	54s
A100	11.7	HunyuanDiT	TensorRT	100	11GB	20s
A100	11.7	HunyuanDiT	Distill+PyTorch	50	13GB	25s
A100	11.7	HunyuanDiT	Distill+TensorRT	50	11GB	10s
3090	11.8	HunyuanDiT	PyTorch	100	14G	98s
3090	11.8	HunyuanDiT	TensorRT	100	14G	40s
3090	11.8	HunyuanDiT	Distill+PyTorch	50	14G	49s
3090	11.8	HunyuanDiT	Distill+TensorRT	50	14G	20s
4090	11.8	HunyuanDiT	PyTorch	100	14G	54s
4090	11.8	HunyuanDiT	TensorRT	100	14G	20s
4090	11.8	HunyuanDiT	Distill+PyTorch	50	14G	27s
4090	11.8	HunyuanDiT	Distill+TensorRT	50	14G	10s

Basically, the requirements for running the models is the same as the original model.

Instructions

The dependencies and installation are basically the same as the original model.

Then download the model using the following commands:

cd HunyuanDiT
# Use the huggingface-cli tool to download the model.
huggingface-cli download Tencent-Hunyuan/Distillation-v1.1 ./pytorch_model_distill.pt --local-dir ./ckpts/t2i/model

Inference

Using Gradio

Make sure you have activated the conda environment before running the following command.

# By default, we start a Chinese UI.
python app/hydit_app.py  --load-key distill 

# Using Flash Attention for acceleration.
python app/hydit_app.py --infer-mode fa --load-key distill   

# You can disable the enhancement model if the GPU memory is insufficient.
# The enhancement will be unavailable until you restart the app without the `--no-enhance` flag. 
python app/hydit_app.py --no-enhance ---load-key distill  

# Start with English UI
python app/hydit_app.py --lang en --load-key distill

Using Command Line

We provide several commands to quick start:

# Prompt Enhancement + Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚" --load-key distill  --infer-steps 50

# Only Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚" --no-enhance --load-key distill  --infer-steps 50

# Only Text-to-Image. Flash Attention mode
python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚" --load-key distill --infer-steps 50

# Generate an image with other image sizes.
python sample_t2i.py --prompt "渔舟唱晚" --image-size 1280 768 --load-key distill  --infer-steps 50

More example prompts can be found in example_prompts.txt

Library: hunyuan-dit
License: other
Languages: en, zh
Created: Jun 12, 2024
Updated: Jun 19, 2024