AnimeFaces256_AEKL

AnimeFaces256-AEKL

This repository contains an AutoencoderKL (AEKL) trained from scratch for latent-space image synthesis experiments on anime face datasets. This model is designed specifically for latent-space synthesis rather than direct image generation.

Model Details

• Architecture: AutoencoderKL (AEKL)
• Input Resolution: 256×256
• Latent Space: (4, 16, 16)
• Components: Encoder–Decoder with KL divergence regularization
• Features: Includes attention mechanisms in latent feature extraction and uses EMA (Exponential Moving Average) weights during training for improved stability.
• VAE Scale Factor: Fixed at 0.04706926643848419 (required for correct latent normalization)
• Author: Yoel Pilier (ORCID) (GitHub)

Intended Purpose

• Latent-space image synthesis research
• KL-regularized autoencoders exploration
• Perceptual learning (LPIPS) evaluation
• Foundation model experimentation
• Attention-based VAE studies

---

Results (Epoch 99)

Reconstruction

Latent Interpolation

---

Usage

Installation & Setup

To train or use the model from the source repository:

git clone https://github.com/YoelPilier/AnimeFaces256_AEKL
cd AnimeFaces256_AEKL
pip install -r requirements.txt

Loading the Model

from Models import VAE
import torch

vae = VAE.from_pretrained(checkpoint="best", use_ema=True)
vae = vae.to("cuda")
vae.eval()

Encoding Images into Latents

with torch.no_grad():
    x = batch_images.to("cuda")
    z_mean, z_log_var = vae.encoder(x)
    z = vae.reparameterize(z_mean, z_log_var)
    latents = z * vae.scaling_factor

Decoding Latents into Images

with torch.no_grad():
    z = latents / vae.scaling_factor
    images = vae.decoder(z)

---

Core VAE Implementation

For reference, the core architecture is structured as follows:

class VAE(nn.Module):
    REPO_ID = "puruchinera/AnimeFaces256_AEKL"

    def __init__(self, input_dim=3, hidden_dim=64, latent_dim=4, compression_factor=4):
        super().__init__()

        self.encoder = Encoder(
            input_dim,
            hidden_dim,
            num_downsamples=compression_factor,
            latent_size=latent_dim,
        )

        self.decoder = Decoder(
            input_dim,
            hidden_dim,
            num_upsamples=compression_factor,
            latent_size=latent_dim,
        )

        self.scaling_factor = 0.04706926643848419
        self.latent_dim = latent_dim

    def forward(self, x):
        z_mean, z_log_var = self.encoder(x)
        z = self.reparameterize(z_mean, z_log_var)
        reconstructed = self.decoder(z)
        return reconstructed, z_mean, z_log_var

    def reparameterize(self, mean, log_var):
        log_var = torch.tanh(log_var) * 5
        std = torch.exp(0.5 * log_var)
        eps = torch.randn_like(std)
        return mean + eps * std

---

Training Details

To train the model yourself using accelerate:

accelerate launch train_vae.py

Hardware & Time

• Framework: PyTorch (trained from scratch)
• Hardware: NVIDIA A6000
• Total Training: 100 epochs
• Epoch Duration: ~1.5 hours
• Total Training Time: ~150 hours

Optimizer & Learning Rate Schedule

optimizer = torch.optim.AdamW(
    model.parameters(),
    lr=3e-4,
    weight_decay=1e-5
)

• Epoch < 74: lr = 3e-4
• Epoch ≥ 74: lr = 3e-5
• Epoch ≥ 85: lr = 5e-6

Losses Evaluated

• Reconstruction loss
• KL Divergence
• LPIPS perceptual loss

---

Final Training Metrics (Epoch 99)

Raw Logs

• Training Metrics Log
• Validation Metrics Log

How to Cite

If you use this repository or its code in your research or projects, please cite it as:

@misc{pilier2026animefaces256aekl,
  author       = {Yoel Pilier},
  title        = {AnimeFaces256-AEKL: AutoencoderKL trained for latent-space image synthesis on anime face datasets},
  year         = {2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/YoelPilier/AnimeFaces256_AEKL}}
}