Downloads · 30 days
53
0% of all-time downloads
Llama-2-7b-hf-gptq-2bit
kaitchup/Llama-2-7b-hf-gptq-2bit
Llama-2-7b-hf-gptq-2bit is a text generation model from kaitchup. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Llama 2 7B quantized in 2-bit with GPTQ.
All-time downloads
25.7K
Public
Parameters
6.7B
2.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
Parameter types
How the weights are stored.
I326.5B · 96%
Try a prompt
Model card
From the Hugging Face model README
Llama 2 7B quantized in 2-bit with GPTQ.
from transformers import AutoModelForCausalLM, AutoTokenizer
from optimum.gptq import GPTQQuantizer
import torch
w = 2
model_path = meta-llama/Llama-2-7b-hf
tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16)
quantizer = GPTQQuantizer(bits=w, dataset="c4", model_seqlen = 4096)
quantized_model = quantizer.quantize_model(model, tokenizer)
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Created
- Jan 19, 2024
- Updated
- Jan 20, 2024