CogVideoX-LoRA-Wallace_and_Gromit

This model has been created using CogVideoX-Factory

This CogVideoX LoRA is created as part of a <b>fan project</b> for <b>research purposes</b> only and is <b>not</b> intended for commercial use. It is based on Wallace and Gromit film clips, which are protected by copyright. Users utilize the model at their own risk. Users are obligated to comply with copyright laws and applicable regulations. The model has been developed for non-commercial purposes, and it is not my intention to infringe on any copyright. I assume no responsibility for any damages or legal consequences arising from the use of the model.

You can use the finetuned model for inference with the following code:

<u><b>Trigger word(s):</b></u>

Use these in your prompt helps providing the style. See example prompt below.

• walgro1
• In the claymation style of Wallace and Gromit

<u><b>Inference:</u></b>

import torch
from diffusers import CogVideoXPipeline
from diffusers import export_to_video

pipe = CogVideoXPipeline.from_pretrained(
    "THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16
).to("cuda")

pipe.load_lora_weights("Cseti/walgro1-2000.safetensors", adapter_name=["cogvideox-lora"])
pipe.set_adapters(["cogvideox-lora"], [1.0])

video = pipe("walgro1. The scene begins with a close-up of Gromit’s face, his expressive eyes filling the frame. His brow furrows slightly, ears perked forward in concentration. The soft lighting highlights the subtle details of his fur, every strand catching the warm sunlight filtering in from a nearby window. His dark, round nose twitches ever so slightly, sensing something in the air, and his gaze darts to the side, following an unseen movement. The camera lingers on Gromit’s face, capturing the subtleties of his expression—a quirked eyebrow and a knowing look that suggests he’s piecing together something clever. His silent, thoughtful demeanor speaks volumes as he watches the scene unfold with quiet intensity. The background remains out of focus, drawing all attention to the sharp intelligence in his eyes and the slight tilt of his head. In the claymation style of Wallace and Gromit.").frames[0]
export_to_video(video, "output.mp4", fps=8)

<u><b>Examples:</b></u>

<b>Prompt:</b> "walgro1. Gromit sits quietly in a cozy living room, the soft glow of a nearby lamp casting warm light across the room. The camera starts with a close-up of his thoughtful expression, his eyes darting toward the side, observing the subtle movement of something off-screen. A clock ticks rhythmically on the wall behind him, creating a steady backdrop to the otherwise silent room. The camera slowly pulls back to reveal the setting: a tidy space with bookshelves filled with old volumes, a comfortable armchair in the corner, and a small coffee table in the center, where a half-finished jigsaw puzzle lies scattered. The atmosphere is calm, almost serene, as Gromit glances toward the puzzle, his curiosity piqued." <video width="720" height="480" controls> <source src="https://huggingface.co/Cseti/CogVideoX-LoRA-Wallace_and_Gromit/resolve/main/WalGro_withLoRA_0000.mp4" type="video/mp4"> Your browser does not support the video tag. </video>

Training

This LoRA was trained on RunPod cloud GPUs.

Support

Producing and sharing this kind of open-source work requires renting cloud GPUs, which gets expensive quickly. If you find it useful and would like me to keep contributing, your support is very much appreciated: