Deep-Incubation

Deep Incubation

This repository contains the pre-trained models for Deep Incubation.

Title:  Deep Incubation: Training Large Models by Divide-and-Conquering
Authors:  Zanlin Ni, Yulin Wang, Jiangwei Yu, Haojun Jiang, Yue Cao, Gao Huang (Corresponding Author)
Institute: Tsinghua University and Beijing Academy of Artificial Intelligence (BAAI)
Publish:   arXiv preprint (arXiv 2212.04129)
Contact:  nzl22 at mails dot tsinghua dot edu dot cn

Models

Data Preparation

• The ImageNet dataset should be prepared as follows:

data
├── train
│   ├── folder 1 (class 1)
│   ├── folder 2 (class 1)
│   ├── ...
├── val
│   ├── folder 1 (class 1)
│   ├── folder 2 (class 1)
│   ├── ...

Citation

If you find our work helpful, please star🌟 this repo and cite📑 our paper. Thanks for your support!

@article{Ni2022Incub,
  title={Deep Incubation: Training Large Models by Divide-and-Conquering},
  author={Ni, Zanlin and Wang, Yulin and Yu, Jiangwei and Jiang, Haojun and Cao, Yue and Huang, Gao},
  journal={arXiv preprint arXiv:2212.04129},
  year={2022}
}

Acknowledgements

Our implementation is mainly based on deit. We thank to their clean codebase.

Contact

If you have any questions or concerns, please send mail to [email protected].