Downloads · 30 days
15
3% of all-time downloads
assistant-llama2-7b-chat-awq
wasertech/assistant-llama2-7b-chat-awq
assistant-llama2-7b-chat-awq is a text generation model from wasertech. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
This model is a quantitized export of wasertech/assistant-llama2-7b-chat using AWQ.
All-time downloads
541
Public
Repo size
7.8 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin3.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Oct 7, 2023
- Updated
- Oct 7, 2023
- SHA
- 21282932
Try a prompt
Datasets
Model card
From the Hugging Face model README
Assistant Llama 2 7B Chat AWQ
This model is a quantitized export of wasertech/assistant-llama2-7b-chat using AWQ.
AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference.
It is also now supported by continuous batching server vLLM, allowing use of Llama AWQ models for high-throughput concurrent inference in multi-user server scenarios.
As of September 25th 2023, preliminary Llama-only AWQ support has also been added to Huggingface Text Generation Inference (TGI).
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- en, fr
- Created
- Oct 7, 2023
- Updated
- Oct 7, 2023