Downloads · 30 days
40
6% of all-time downloads
Kooten/WinterGoddess-1.4x-70B-L2-IQ2-GGUF
WinterGoddess-1.4x-70B-L2-IQ2-GGUF is a machine learning model from Kooten. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
IQ2-GGUF quants of Sao10K/WinterGoddess-1.4x-70B-L2
Downloads · 30 days
40
6% of all-time downloads
All-time downloads
628
Public
Repo size
38.6 GB
Likes
1
Public
Click a slice to open those files.
.gguf38.6 GB · 100%
From the Hugging Face model README
IQ2-GGUF quants of Sao10K/WinterGoddess-1.4x-70B-L2
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
NOTE: Currently you will need experimental branches of Koboldcpp or Ooba for this to work.
Regular GGUF Quants: Here
### Instruction:
<Prompt>
### Response:
OR
### Instruction:
<Prompt>
### Input:
<Insert Context Here>
### Response:
Kooten on discord