complete-the-look

complete the look - fashion compatibility model

scene-based complementary product recommendation model. given an outfit photo, find products that match the style.

model description

trained on the shop the look dataset with fashion SigLIP embeddings.

architecture

• input: fashion SigLIP embeddings (768-dim)
• scene encoder: MLP with residual blocks → 256-dim
• product encoder: MLP with category embeddings → 256-dim
• training: contrastive + triplet loss with hard negative mining

performance

files

• best_model_v2.pt - model weights (37MB)
• product_embeddings.pkl - pre-computed product embeddings (110MB, 36K products)
• scene_embeddings.pkl - pre-computed scene embeddings (87MB, 29K scenes)

usage

import torch
import pickle

checkpoint = torch.load("best_model_v2.pt", map_location="cpu", weights_only=False)
# see github repo for full code

with open("product_embeddings.pkl", "rb") as f:
    product_embeddings = pickle.load(f)

code

full training code and web demo: github repo

citation

based on:

@inproceedings{kang2019complete,
  title={Complete the Look: Scene-based Complementary Product Recommendation},
  author={Kang, Wang-Cheng and Kim, Eric and Leskovec, Jure and Rosenberg, Charles and McAuley, Julian},
  booktitle={CVPR},
  year={2019}
}