Downloads · 30 days
0
Wizard-Vicuna-7B-Uncensored-ggml
localmodels/Wizard-Vicuna-7B-Uncensored-ggml
Wizard-Vicuna-7B-Uncensored-ggml is a machine learning model from localmodels. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
From: https://huggingface.co/ehartford/Wizard-Vicuna-7B-Uncensored
Access
Public
Updated Jul 12, 2023
Repo size
86.6 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin60.1 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 12, 2023
- Updated
- Jul 12, 2023
- SHA
- 84a3de35
Model card
From the Hugging Face model README
Wizard Vicuna 7B Uncensored ggml
From: https://huggingface.co/ehartford/Wizard-Vicuna-7B-Uncensored
Original llama.cpp quant methods: q4_0, q4_1, q5_0, q5_1, q8_0
Quantized using an older version of llama.cpp and compatible with llama.cpp from May 19, commit 2d5db48.
k-quant methods: q2_K, q3_K_S, q3_K_M, q3_K_L, q4_K_S, q4_K_M, q5_K_S, q6_K
Quantization methods compatible with latest llama.cpp from June 6, commit 2d43387.
Files
| Name | Quant method | Bits | Size | Max RAM required | Use case |
|---|---|---|---|---|---|
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q2_K.bin | q2_K | 2 | 2.80 GB | 5.30 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors. |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q3_K_L.bin | q3_K_L | 3 | 3.55 GB | 6.05 GB | New k-quant method. Uses GGML_TYPE_Q5_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q3_K_M.bin | q3_K_M | 3 | 3.23 GB | 5.73 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q3_K_S.bin | q3_K_S | 3 | 2.90 GB | 5.40 GB | New k-quant method. Uses GGML_TYPE_Q3_K for all tensors |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_K_M.bin | q4_K_M | 4 | 4.05 GB | 6.55 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q4_K |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_K_S.bin | q4_K_S | 4 | 3.79 GB | 6.29 GB | New k-quant method. Uses GGML_TYPE_Q4_K for all tensors |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q5_K_M.bin | q5_K_M | 5 | 4.77 GB | 7.27 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q5_K_S.bin | q5_K_S | 5 | 4.63 GB | 7.13 GB | New k-quant method. Uses GGML_TYPE_Q5_K for all tensors |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q6_K.bin | q6_K | 6 | 5.53 GB | 8.03 GB | New k-quant method. Uses GGML_TYPE_Q8_K - 6-bit quantization - for all tensors |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_0.bin | q4_0 | 4 | 3.79 GB | 6.29 GB | Original llama.cpp quant method, 4-bit. |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_1.bin | q4_1 | 4 | 4.21 GB | 6.71 GB | Original llama.cpp quant method, 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q5_0.bin | q5_0 | 5 | 4.63 GB | 7.13 GB | Original llama.cpp quant method, 5-bit. Higher accuracy, higher resource usage and slower inference. |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q5_1.bin | q5_1 | 5 | 5.06 GB | 7.56 GB | Original llama.cpp quant method, 5-bit. Even higher accuracy, resource usage and slower inference. |
| Wizard-Vicuna-7B-Uncensored.ggmlv3.q8_0.bin | q8_0 | 8 | 7.16 GB | 9.66 GB | Original llama.cpp quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |
- Created
- Jul 12, 2023
- Updated
- Jul 12, 2023