Qwanima-vae

Qwanima-vae

This is a decoder-only finetune for the Qwen-Image VAE used by Anima. It is directly inspired by spacepxl's 2x Wan VAE upscaler, adapted to Qwen-Image and Anima using the methodology described by spacepxl. More detail about the original method is available on that model card.

The motivation behind this project is mostly the same as spacepxl's. The Qwen-Image VAE produces a rather unpleasant dithered look. Since it was also made with photographs in mind, it can reconstruct fine detail as noisy or incoherent texture. This decoder finetune aims to reduce that and produce a cleaner image suited to Anima generations.

The released checkpoint is the 45k EMA from the best 256px run. The encoder is unchanged from Qwen-Image and only the decoder was finetuned, so this uses the same 16-channel latent format and produces images at the same resolution as the original VAE.

Comparison

Model

Inference is the same as with a normal VAE:

Anima latent
    ↓
Qwen-Image VAE decoder (Anima finetuned)
    ↓
image

Training

Training looked roughly like this:

training image
    ↓
frozen Qwen VAE encoder
    ↓
clean latent
    ↓
degradation via a learned proxy model
    ↓
trainable Qwen VAE decoder
    ↓
reconstruction

The training data used a split of 75% Booru Essence and 25% Cleveland Museum of Art images. Booru Essence covers the anime part of the distribution, while the museum data adds paintings, physical media, and other fine texture to prevent the decoder from forgetting those details.

The final run used:

• 256x256 training crops
• Frozen Qwen-Image encoder
• Anima degradation proxy, applied with probability 0.8 over timesteps 0–0.12
• MSE loss at 1.0
• DINOv3 ViT-B feature loss at 8.0, using patch tokens from all layers
• Anti-aliased PatchGAN with LSGAN loss at 0.25

I settled on MSE at weight 1.0 because that gave the GAN enough freedom to produce details while still anchoring the result to the pixel target. I previously used L1, but it anchored the reconstruction too aggressively and hampered the GAN's ability to add detail in my shoddy testing.

I won't pretend these are the optimal settings. This is simply what I ended up with in the end.

45k was selected because it looked the best. Was going to train it to 100k steps switching to 512px at 50k but it was steadily degrading as training continued. I think GAN weight might've been too high.

Anti-aliased PatchGAN

PatchGAN judges a grid of small local patches instead of reducing the entire image to a single real/fake score. That makes it useful for texture, but its stride-2 convolutions can also alias high frequencies while downsampling. The discriminator can then become sensitive to pixel-grid phase and mistake regular dithering or checkerboard patterns for convincing detail.

To make that shortcut less attractive, both the real and generated images are low-pass filtered immediately before each of PatchGAN's three stride-2 stages. The filter is a fixed, normalized 3x3 binomial kernel:

[1, 2, 1]ᵀ × [1, 2, 1] / 16

This band-limits the signal before subsampling, encouraging the discriminator to judge coherent local structure instead of phase-locked pixel energy. The filtering only exists inside the discriminator: it does not blur the saved decoder output or the images used by MSE and DINO.

The discriminator itself uses four spectral-normalized layers with 64 base channels. GAN training began at step 1,000, ramped to weight 0.25 over 2,000 steps, and used LSGAN.

Diffusers

diffusion_pytorch_model.safetensors is a standard Diffusers checkpoint. Use it with:

import torch
from diffusers import AutoencoderKLQwenImage

vae = AutoencoderKLQwenImage.from_pretrained(
    "PATH_OR_REPO_ID",
    torch_dtype=torch.bfloat16,
).to("cuda")

ComfyUI

A converted BF16 version for ComfyUI is located here:

comfyui/qwen_vae_anima.safetensors

Copy it into ComfyUI/models/vae/, refresh or restart ComfyUI, and load it with the normal Load VAE node.

2x upscaling

I mostly focused on 1x decoding for this project, so that is what I am releasing. This VAE does not change the output resolution and a 1024px Anima latent still decodes to a 1024px image.

Files

.
├── config.json
├── diffusion_pytorch_model.safetensors
├── comfyui/
│   └── qwen_vae_anima.safetensors
├── comparison/
│   └── preview.png
└── LICENSE

diffusion_pytorch_model.safetensors is the FP32 Diffusers checkpoint.

comfyui/qwen_vae_anima.safetensors is the converted BF16 ComfyUI checkpoint.

Limitations

Since this was trained mostly on anime and illustration it won't provide good results on images far outside that distribution. Photos and textured media can be oversmoothed or otherwise altered, and difficult high-frequency regions may develop grain or speckles. For a 2x VAE with broader material coverage, use spacepxl's Wan2.1 VAE upscaler.

License

Released under the Apache License 2.0.