Downloads · 30 days
8
22% of all-time downloads
llama2-7b-2to4-int4
smk2295/llama2-7b-2to4-int4
llama2-7b-2to4-int4 is a machine learning model from smk2295. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama2.
meta-llama/Llama-2-7b-hf compressed with ASQ (ADMM joint prune + quantize), exported to compressed-tensors marlin-24 (2:4 semi-structured sparsity on a per-group INT4 grid, group size 128). Loads in vLLM via the Spars…
All-time downloads
36
Public
Parameters
1.1B
3.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 100%
Parameter types
How the weights are stored.
I32405M · 36%
Base models
Model card
From the Hugging Face model README
Llama-2-7b — 2:4 sparse + INT4 (W4A16), Marlin-24
meta-llama/Llama-2-7b-hf compressed with ASQ (ADMM joint prune + quantize),
exported to compressed-tensors marlin-24 (2:4 semi-structured sparsity on a
per-group INT4 grid, group size 128). Loads in vLLM via the Sparse-Marlin kernel.
- Sparsity: 2:4 (50%) · Quant: INT4 W4A16, per-group g128 · Method: global ADMM, z_alt (x+u)
- WikiText-2 ppl (pre-export): 9.595 · C4 ppl: 12.037
from vllm import LLM
llm = LLM("smk2295/llama2-7b-2to4-int4")
- Type
- llama
- License
- llama2
- Created
- Sep 23, 2026
- Updated
- Sep 23, 2026