Downloads · 30 days
0
viddle-pix2pix-animatediff
viddle/viddle-pix2pix-animatediff
viddle-pix2pix-animatediff is a machine learning model from viddle. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a fine-tuned version of the AnimateDiff V2 model using the InstructPix2Pix as the T2I model.
Access
Public
Updated Sep 28, 2023
Repo size
1.7 GB
Likes
7
Public
Repo makeup
Click a slice to open those files.
.ckpt1.7 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 28, 2023
- Updated
- Sep 28, 2023
- SHA
- e54d824e
Model card
From the Hugging Face model README
InstructPix2Pix for AnimateDiff
This model is a fine-tuned version of the AnimateDiff V2 model using the InstructPix2Pix as the T2I model.
Inference
You need use a pipeline built for InstructPix2Pix and AnimateDiff inference. And example is here
Using the pipeline, InstructPix2Pix T2I model and this motion module, you can generate animations that use an init image faithfully.
Init Image
Make her smile
Make her on fire
Training
The training procedure was modified to pass the first frame of all videos to the extra channels of the InstructPix2Pix model. The model was trained 4500 steps with a learning rate of 5e-7 and 5500 steps at 1e-7, with a batch size of 1.
LICENSE
Creative OpenRAIL License
- Created
- Sep 28, 2023
- Updated
- Sep 28, 2023