AGSM

Alignment-Guided Score Matching (AGSM)

This repository hosts the released AGSM soft-token checkpoints for:

• SD3: sd3/soft_tokens.pth, sd3/soft_t_tokens.pth
• SD1.5: sd1.5/soft_tokens.pth, sd1.5/soft_t_tokens.pth
• SDXL: sdxl/soft_tokens.pth, sdxl/soft_t_tokens.pth

AGSM is a lightweight, reward-free post-training method for improving text-image alignment in diffusion models. It requires no external reward model, no full denoising rollout, and no (x_0) approximation.

Usage

This repository contains AGSM token checkpoints, not the full base diffusion models. The GitHub repository already includes the released checkpoints, so the simplest path is:

git clone https://github.com/jaayeon/AGSM.git
cd AGSM

DATADIR=/path/to/datasets \
MODEL=sd3 \
scripts/sample_coco.sh

If you want to use the Hugging Face copy instead, download it separately and point CHECKPOINT_DIR to the downloaded model folder:

huggingface-cli download jaayeon/AGSM --local-dir checkpoints/agsm

DATADIR=/path/to/datasets \
MODEL=sd3 \
CHECKPOINT_DIR=checkpoints/agsm/sd3 \
scripts/sample_coco.sh

Use MODEL=sd1.5 with CHECKPOINT_DIR=checkpoints/agsm/sd1.5, or MODEL=sdxl with CHECKPOINT_DIR=checkpoints/agsm/sdxl.

Code, training scripts, and evaluation instructions are available at: https://github.com/jaayeon/AGSM

Project page: https://jaayeon.github.io/AGSM/

Paper: https://arxiv.org/abs/2605.30038

Citation

@article{lee2026alignment,
  title={Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models},
  author={Lee, Jaa-Yeon and Hong, Yeobin and Kwon, Taesung and Ye, Jong Chul},
  journal={arXiv preprint arXiv:2605.30038},
  year={2026}
}