Downloads · 30 days
0
gptq-w4-gs32-sparse-compressed-oc14336-ic4096
vuiseng9/gptq-w4-gs32-sparse-compressed-oc14336-ic4096
gptq-w4-gs32-sparse-compressed-oc14336-ic4096 is a machine learning model from vuiseng9. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repo contains serialized blobs of an up projection layer of llama3-8B (oc=14336, ic=4096). The linear layer has been quantized (GPTQ W4 Sym with group size 32) and sparsified by 50%.
Access
Public
Updated Jul 4, 2024
Repo size
446 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth405 MB · 91%
At a glance
- Access
- Public
- Created
- Jul 4, 2024
- Updated
- Jul 4, 2024
- SHA
- cfb9114a
Model card
From the Hugging Face model README
This repo contains serialized blobs of an up projection layer of llama3-8B (oc=14336, ic=4096). The linear layer has been quantized (GPTQ W4 Sym with group size 32) and sparsified by 50%.
├── sparse_w4
│ ├── linear_bitmap_int32.bin
│ ├── linear_compressed_qweight_int32.bin
│ ├── linear_nnz_int16.bin
│ ├── linear_scales_float16.bin
│ └── linear_zeros_int32.bin
Usage
The following script shows how to process the blobs in python. It shows unpacking, zero location recovery, as well as weight dequantization process.
python unpack_blobs.py
you can ignore
internal/
- Created
- Jul 4, 2024
- Updated
- Jul 4, 2024