Downloads · 30 days
161
9% of all-time downloads
llama-v2-2bit-gguf
ikawrakow/llama-v2-2bit-gguf
llama-v2-2bit-gguf is a machine learning model from ikawrakow. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains 2-bit quantized LLaMA-v2 models in GGUF format for use with llama.cpp. All tensors are quantized with Q2K, except for output.weight, which is Q6K, and, in the case of LLaMA-v2-70B, attnv, whic…
All-time downloads
1.8K
Public
Repo size
29.6 GB
Likes
8
Public
Repo makeup
Click a slice to open those files.
.gguf29.6 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Dec 6, 2023
- Updated
- Dec 6, 2023
- SHA
- d38bb73d
Model card
From the Hugging Face model README
This repository contains 2-bit quantized LLaMA-v2 models in GGUF format for use with llama.cpp.
All tensors are quantized with Q2_K, except for output.weight, which is Q6_K, and, in the case of LLaMA-v2-70B, attn_v, which is Q4_K.
The quantized models differ from the standard llama.cpp 2-bit quantization in two ways:
- These are actual 2-bit quantized models instead of the mostly 3-bit quantization provided by the standard
llama.cppQ2_Kquantization method - The models were prepared with a refined (but not yet published) k-quants quantization approach
- License
- apache-2.0
- Created
- Dec 6, 2023
- Updated
- Dec 6, 2023