grasprune_llama2-13b_ratio0.2

Llama-2-13B — original GRASPrune, pruning ratio 0.2

This is a structurally pruned derivative of meta-llama/Llama-2-13b-hf.

Model provenance

This checkpoint was produced by the original GRASPrune pruning pipeline in methods/GRASPrune, not by new_method_2. The local new_method_2 code was used only for downstream evaluation and did not produce or alter the pruned weights.

• Method: GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Models
• Original implementation: ZiY-Wang/GRASPrune
• Upstream source revision recorded with this release: 666f4f04ec9e5e3218904df4d0935841bbd570fc
• Base model: meta-llama/Llama-2-13b-hf
• Target pruning ratio: 0.2
• GRASPrune retained budget: keep_ratio=0.8
• Export dtype/format: torch.bfloat16, materialized state dict
• Rescale compensation: rescale_alpha=0.5

The 20% ratio is the target reduction of GRASPrune's global prunable structural budget across FFN intermediate channels and attention KV groups. It is not a uniform per-layer sparsity ratio or a file-size reduction ratio.

Files and loading

This is not a standard Transformers save_pretrained directory and cannot be loaded directly with AutoModelForCausalLM.from_pretrained().

• pruned_state_dict.safetensors: materialized pruned weights
• meta.json: layer-specific shapes required to rebuild the architecture
• layer_mask_report.csv: layer-wise retention report
• provenance.json: release provenance and artifact identity
• SHA256SUMS: checksum for the weight file

Clone the original GRASPrune repository, run from its root, and use its rebuild.py loader. Loading also requires authorized access to the gated Llama-2-13B base model:

import os
import torch
from huggingface_hub import snapshot_download
from rebuild import load_pruned_model

checkpoint_dir = snapshot_download("LiamCarter/grasprune_llama2-13b_ratio0.2")
model, tokenizer, meta = load_pruned_model(
    model_id="meta-llama/Llama-2-13b-hf",
    state_dict_path=os.path.join(checkpoint_dir, "pruned_state_dict.safetensors"),
    meta_path=os.path.join(checkpoint_dir, "meta.json"),
    torch_dtype=torch.bfloat16,
    device="cuda:0",
    local_only=False,
)

Local evaluation

The weights were evaluated read-only with the local new_method_2 evaluation flow. Percentages:

These are local benchmark results, not upstream GRASPrune claims.

Intended use, limitations, and license

This checkpoint is intended for structured-pruning and benchmark research. It has not been validated for production deployment, safety, factual reliability, bias, multilingual robustness, or long-context behavior. Performance can be lower than the parent model.

This is a derivative of Llama 2 and remains subject to the Llama 2 Community License and Acceptable Use Policy. Llama 2 is licensed under the LLAMA 2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Users must separately obtain access to the gated base model.