Diffusion-DDIM

Diffusion-DDIM — DDIM Sampler on CIFAR-10

An extension of the DDPM-CIFAR10 model that adds a Denoising Diffusion Implicit Models (DDIM) sampler for fast, deterministic inference. The underlying U-Net is trained identically to the DDPM variant; only the sampling procedure differs.

Model Description

DDIM (Song et al., 2020) reformulates the reverse diffusion process as a non-Markovian chain, allowing the model to sample in far fewer steps than standard DDPM while maintaining high image quality. The same noise-predicting U-Net trained with DDPM can be used directly with the DDIM sampler without retraining.

DDIM vs DDPM

Architecture

Identical U-Net to Diffusion-CIFAR10: ResBlocks with GroupNorm, self-attention at the bottleneck, sinusoidal time embeddings, strided Conv2d downsampling, nearest-neighbour upsampling.

Diffusion Process

• Trainer: GaussianDiffusionTrainer — standard DDPM noise prediction training
• Sampler: GaussianDiffusionSampler with DDIM update rule
• Schedule: Linear beta schedule
• Loss: MSE between predicted and actual noise (epsilon parameterization)

Training Details

Sample Images

Generated samples at different DDIM step counts are included in SampledImgs/:

Repository Contents

How to Use

import torch
from model import UNet
from diffusion import GaussianDiffusionSampler

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = UNet(T=1000, ch=128, ch_mult=[1,2,2,2], attn=[1], num_res_blocks=2, dropout=0.1)
model.load_state_dict(torch.load("Checkpoints/ckpt_199.pth", map_location=device))
model.eval().to(device)

# DDIM sampler — set T to desired number of inference steps
sampler = GaussianDiffusionSampler(beta_1=1e-4, beta_t=0.02, model=model, T=200).to(device)

with torch.no_grad():
    x_T = torch.randn(16, 3, 32, 32, device=device)
    samples = sampler(x_T)

References

• Ho et al. (2020). Denoising Diffusion Probabilistic Models
• Song et al. (2020). Denoising Diffusion Implicit Models

License

MIT