Downloads · 30 days
22
2% of all-time downloads
Llama-2-7b-gptq-2bit
kaitchup/Llama-2-7b-gptq-2bit
Llama-2-7b-gptq-2bit is a text generation model from kaitchup. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is Meta's Llama 2 7B quantized in 2-bit using AutoGPTQ from Hugging Face Transformers.
All-time downloads
1.1K
Public
Parameters
6.7B
4.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin2.3 GB · 50%
Parameter types
How the weights are stored.
I326.5B · 96%
Try a prompt
Model card
From the Hugging Face model README
Model Card for Model ID
This is Meta's Llama 2 7B quantized in 2-bit using AutoGPTQ from Hugging Face Transformers.
Model Details
Model Description
<!-- Provide a longer summary of what this model is. -->- Developed by: The Kaitchup
- Model type: Causal (Llama 2)
- Language(s) (NLP): English
- License: Apache 2.0, Llama 2 license agreement
Model Sources
The method and code used to quantize the model are explained here: Quantize and Fine-tune LLMs with GPTQ Using Transformers and TRL
Uses
This model is pre-trained and not fine-tuned. You may fine-tune it with PEFT using adapters. Note that the 2-bit quantization significantly decreases the performance of Llama 2.
Other versions
Model Card Contact
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Aug 29, 2023
- Updated
- Sep 11, 2023