Downloads · 30 days
3
4% of all-time downloads
am00r/Llama-3.2-3B-wanda-sp66-GGUF
Llama-3.2-3B-wanda-sp66-GGUF is a text generation model from am00r. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as llama3.2.
meta-llama/Llama-3.2-3B pruned to 0.66 unstructured sparsity with Wanda, then converted to GGUF f16.
Downloads · 30 days
3
4% of all-time downloads
All-time downloads
84
Public
Repo size
6.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf6.4 GB · 100%
From the Hugging Face model README
meta-llama/Llama-3.2-3B pruned to 0.66 unstructured sparsity with Wanda, then converted to GGUF f16.
Pruning is applied to the decoder linear layers only, so whole-model sparsity is lower than the target. The ratio was chosen to bring the effective parameter count close to Llama 3.2 1B.
Unstructured pruning zeroes weights rather than removing them: this file is the same size as the dense f16 model and llama.cpp has no sparse kernels for it, so expect no speedup and no memory saving from the sparsity alone.
llama-cli -m llama3.2-3b-wanda-66-f16.gguf -p "Hello"
MSc thesis artifact on compression of small language models. Built with Llama, under the Llama 3.2 Community License.