Downloads · 30 days
61
14% of all-time downloads
Koshkasa/TheDrummer_Snowpiercer-15B-v4-IQ5_K-GGUF
TheDrummer_Snowpiercer-15B-v4-IQ5_K-GGUF is a text generation model from Koshkasa. Use it when you need the model to write or continue text. It is set up for ik_llama.cpp. The card lists the license as mit.
The goal: Q5KM+ quality (allegedly) at Q5KS size using SOTA quant types from ikllama.cpp. The result: Mixed precision quantization of TheDrummer/Big-Alice-28B-v1 utilizing IQ5K for ffn and embeddings, and IQ6K for out…
Downloads · 30 days
61
14% of all-time downloads
All-time downloads
434
Public
Repo size
10.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf10.8 GB · 100%
From the Hugging Face model README
The goal: Q5_K_M+ quality (allegedly) at Q5_K_S size using SOTA quant types from ik_llama.cpp.
The result: Mixed precision quantization of TheDrummer/Big-Alice-28B-v1 utilizing IQ5_K for ffn and embeddings, and IQ6_K for output and attention.
quantized with ik_llama.cpp build: 9d07d868
imatrix by bartowski
incompatible with mainline llama.cpp
| Layer | Dims | Dims | Quant |
|---|---|---|---|
| token_embd | 5120 | 131072.0 | iq5_k |
| blk | 50 | ||
| attn_k | 5120 | 1024 | iq6_k |
| attn_norm | 5120 | 1 | f32 |
| attn_q | 5120 | 4096 | iq6_k |
| attn_v | 5120 | 1024 | iq6_k |
| attn_output | 4096 | 5120 | iq6_k |
| ffn_down | 14336 | 5120 | iq5_k |
| ffn_gate | 5120 | 14336 | iq5_k |
| ffn_norm | 5120 | 1 | f32 |
| ffn_up | 5120 | 14336 | iq5_k |
| output | 5120 | 131072 | iq6_k |
| output_norm | 5120 | 1 | f32 |
SillyTilly - for the base model.
TheDrummer - for the finetuned model.
bartowski - for the imatrix + the myriad of quants we all benefit from.
ikawrakow and contributors of ik_llama.cpp - I probably misused your ingenious creation.