facexformer

FaceXFormer Model Card

<div align="center">

Project Page | Paper (ArXiv) | Code

</div>

Introduction

FaceXFormer is an end-to-end unified model capable of handling a comprehensive range of facial analysis tasks such as face parsing, landmark detection, head pose estimation, attributes prediction, age/gender/race estimation, facial expression recognition and face visibility prediction.

<div align="center"> <img src='assets/intro.png'> </div>

Model Details

FaceXFormer is a transformer-based encoder-decoder architecture where each task is treated as a learnable token, enabling the integration of multiple tasks within a single framework.

<div align="center"> <img src='assets/main_archi.png'> </div>

Usage

The models can be downloaded directly from this repository or using python:

from huggingface_hub import hf_hub_download

hf_hub_download(repo_id="kartiknarayan/facexformer", filename="ckpts/model.pt", local_dir="./")

Citation

@article{narayan2024facexformer,
  title={FaceXFormer: A Unified Transformer for Facial Analysis},
  author={Narayan, Kartik and VS, Vibashan and Chellappa, Rama and Patel, Vishal M},
  journal={arXiv preprint arXiv:2403.12960},
  year={2024}
}

Please check our GitHub repository for complete inference instructions.