SelfCite-8B-from-CC

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

Paper: https://arxiv.org/abs/2502.09604
Authors: Yung-Sung Chuang†, Benjamin Cohen-Wang†, Shannon Zejiang Shen†, Zhaofeng Wu†, Hu Xu‡, Xi Victoria Lin‡, James Glass†, Shang-Wen Li‡, Wen-tau Yih‡
† Massachusetts Institute of Technology, ‡ Meta AI

This model is a reproduction of the SelfCite 8B SimPO fine-tuned model, trained by running the official training scripts from the SelfCite repository.
The model is initialized from Llama-3.1-8B-Instruct, and then first trained with SFT data from ContextCite (256 calls). This is the fully self-supervised setting experiment in the SelfCite paper.
Please refer to our GitHub repository for usage and more details: https://github.com/facebookresearch/SelfCite

Citation

Please cite our paper as well as LongCite if they are helpful to your work!

@inproceedings{chuang2025selfcite,
  title={SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models},
  author={Yung-Sung Chuang and Benjamin Cohen-Wang and Shannon Zejiang Shen and Zhaofeng Wu and Hu Xu and Xi Victoria Lin and James Glass and Shang-Wen Li and Wen-tau Yih},
  journal={arXiv preprint arXiv:2502.09604},
  year={2025}
}

@article{zhang2024longcite,
  title = {LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA} 
  author={Jiajie Zhang and Yushi Bai and Xin Lv and Wanjun Gu and Danqing Liu and Minhao Zou and Shulin Cao and Lei Hou and Yuxiao Dong and Ling Feng and Juanzi Li},
  journal={arXiv preprint arXiv:2409.02897},
  year={2024}
}