Downloads · 30 days
41
0% of all-time downloads
Sheared-LLaMA-1.3B-Pruned
princeton-nlp/Sheared-LLaMA-1.3B-Pruned
Sheared-LLaMA-1.3B-Pruned is a text generation model from princeton-nlp. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Paper: https://arxiv.org/pdf/2310.06694.pdf Code: https://github.com/princeton-nlp/LLM-Shearing Models: Sheared-LLaMA-1.3B, Sheared-LLaMA-2.7B Pruned Models without Continued Pre-training: Sheared-LLaMA-1.3B-Pruned, S…
All-time downloads
9.5K
Public
Repo size
5.4 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.bin2.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Jan 23, 2024
- Updated
- Jan 23, 2024
- SHA
- 97faa597
Try a prompt
Model card
From the Hugging Face model README
Paper: https://arxiv.org/pdf/2310.06694.pdf
Code: https://github.com/princeton-nlp/LLM-Shearing
Models: Sheared-LLaMA-1.3B, Sheared-LLaMA-2.7B
Pruned Models without Continued Pre-training: Sheared-LLaMA-1.3B-Pruned, Sheared-LLaMA-2.7B-Pruned
Instruction-tuned Models: Sheared-LLaMA-1.3B-ShareGPT, Sheared-LLaMA-2.7B-ShareGPT
License: Must comply with license of Llama2 since it's a model derived from Llama2.
Sheared-LLaMA-1.3B-Pruned is the model pruned from meta-llama/Llama-2-7b-hf without continued pre-training. We used roughly 0.4B tokens to perform the pruning experiment. This model could be a good use to study
- effective data mixtures for continued pre-training
- comparisons to other pruning techniques
- extensive evaluations to understand how pruning affects knowledge and reasoning capabilities of LLMs
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Jan 23, 2024
- Updated
- Jan 23, 2024