Anima-GGUF

GGUF models of ANIMA

How to use

• Update ComfyUI to v0.14.1 or above. (You don't need custom script.)
• Download GGUF you want. (Recommend at least Q5_K_M)
• Use ComfyUI-GGUF custom node to load the model.

Generation speed

Tested on

• RTX5090(400W), ComfyUI with --fast option and Patch Sage Attention KJ node(AUTO).
• 832x1216, cfg 5.0, 50steps

Sample

anima-preview3-base

anima-preview2

anima-preview

How to reproduce

1. Convert BF16 model to FP32

import torch
import safetensors.torch
import os
import sys

def convert_to_fp32(input_path, output_path):
    state_dict = safetensors.torch.load_file(input_path)

    new_state_dict = {}
    for key, tensor in state_dict.items():
        print(f"{key} ({tensor.dtype}) -> torch.float32")
        new_tensor = tensor.to(torch.float32)
        new_state_dict[key] = new_tensor

    safetensors.torch.save_file(new_state_dict, output_path)
    print(f"output_path: {output_path}")

if __name__ == "__main__":
    assert len(sys.argv) == 3, f"usage: {sys.argv[0]} SOURCE TARGET"
    input_path, output_path = sys.argv[1:3]

    convert_to_fp32(input_path, output_path)

1. Read this manual.
2. make F32 GGUF using https://github.com/city96/ComfyUI-GGUF/blob/main/tools/convert.py#L258
3. Run llama-quantize.