Downloads · 30 days
30
40% of all-time downloads
Linear_pruned_RedPajama3B
Advaith28/Linear_pruned_RedPajama3B
Linear_pruned_RedPajama3B is a text generation model from Advaith28. Use it when you need the model to write or continue text. It is set up for transformers.
33% pruning on RedPajama 3B linear layers
All-time downloads
75
Public
Parameters
2.8B
11.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3 GB · 100%
Parameter types
How the weights are stored.
I82.5B · 91%
Try a prompt
Model card
From the Hugging Face model README
33% pruning on RedPajama 3B linear layers
The pruned layers are:
- attention linear layers (query, key, value computation)
- attention dense layer
- MLP layers
Pruning is done in all decoder modules. Pruning is unstructured magnitude pruning
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neox
- Created
- Nov 12, 2023
- Updated
- Nov 12, 2023