prosparse-llama-2-7b-predictor

ProSparse-LLaMA-2-7B-Predictor

• Model Creator: THUNLP, ModelBest, and PowerInfer

This repository provides a group of sparsity predictors serving for SparseLLM/ProSparse-LLaMA-2-7B.

Note: The folder predictors_clip15 contains the activation predictors trained with data only reserving top 15% activation values.

Citation

Please kindly cite using the following BibTeX:

@article{song2024prosparse,
  title={{ProSparse}: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models},
  author={Song, Chenyang and Han, Xu and Zhang, Zhengyan and Hu, Shengding and Shi, Xiyu and Li, Kuai and Chen, Chen and Liu, Zhiyuan and Li, Guangli and Yang, Tao and Sun, Maosong},
  year={2024},
  journal={arXiv preprint arXiv:2402.13516},
  url={https://arxiv.org/pdf/2402.13516.pdf}
}