Downloads · 30 days
184
3% of all-time downloads
NexesQuants/alchemonaut_QuartetAnemoi-70B-iMat.GGUF
alchemonaut_QuartetAnemoi-70B-iMat.GGUF is a machine learning model from NexesQuants. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
GGUF quants for : https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001
Downloads · 30 days
184
3% of all-time downloads
All-time downloads
7K
Public
Repo size
319 GB
Likes
11
Public
Click a slice to open those files.
.gguf304 GB · 100%
From the Hugging Face model README
GGUF quants for : https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001
Available : Q3_K_M, IQ3_XXS. Otw : IQ2_XS
I recommand you folks to try this model, because it's quite an efficient merge of Miqu, WinterGoddess, AuroraNights, and XWin.
The Theta Rope 1,000,000 of Miqu, hence the 32k context, is functional up to 16k accordingly to my tests, and probably above (I need a smaller quant to test, which is otw).
To use it with a quantized KV cache for a higher context, here's KoboldCPP Frankenstein version with several different KV cache quantization lebels to chose from : https://github.com/Nexesenex/kobold.cpp/releases