slot-rae-dinov3-b-coco

Slot-RAE (DINOv3-B, COCO)

Checkpoint of Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders (under review).

Slot-RAE learns 7 object slots (768-d, plus 1 global token) on top of a frozen DINOv3 ViT-B/16 encoder and trains a slot-conditioned Diffusion Transformer (Base, with a wide DDT head and a REPA head) from scratch to regenerate the DINOv3 features. A frozen RAEv2 decoder maps generated features to pixels. This enables zero-shot object removal, insertion and swapping.

• 🌐 Project page: https://slot-rae.github.io/
• 💻 Code: https://github.com/alexcbb/slot-rae

Files

Usage

from huggingface_hub import hf_hub_download
from slot_rae import load_slot_rae  # pip install -e . from the code repository

ckpt = hf_hub_download("Beegbrain/slot-rae-dinov3-b-coco", "slot_rae_dinov3_b_coco.ckpt")
model = load_slot_rae(ckpt, device="cuda")      # RAE weights: see the code README (RAE_ROOT)
out = model(images)                              # ImageNet-normalized (B, 3, 256, 256)
out["slots"], out["masks"], out["recon_image"]

Results (COCO val2017)

License

CC BY-NC 4.0, inherited from RAEv2.