Downloads · 30 days
162
3% of all-time downloads
KeinNiemand/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated-IK_GGUF
Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated-IK_GGUF is a text generation model from KeinNiemand. Use it when you need the model to write or continue text.
Downloads · 30 days
162
3% of all-time downloads
All-time downloads
5.4K
Public
Repo size
592 GB
Likes
4
Public
Click a slice to open those files.
.gguf591 GB · 100%
From the Hugging Face model README
CRITICAL COMPATIBILITY WARNING
These are iqk format quantizations and are EXCLUSIVE to the ik_llama.cpp fork.
They will NOT work on mainline llama.cpp, standard LM Studio, standard Text Generation WebUI, or KoboldCPP.
You must compile and run this using ikawrakow's llama.cpp fork, or a UI where you have manually swapped the backend to an ik_llama.cpp build.
This repository contains custom, mixed-precision ik_llama.cpp GGUF quantizations for OpenYourMind/Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated, a Kimi-K2.6 distilled, healed, abliterated Qwen3.5 122B A10B MoE model.
These quants use different precision levels for different layer types, keeping attention, SSM, shared expert, output, and MTP/NextN tensors at higher precision while compressing the routed experts, which make up the bulk of the model's size.
These quantizations have NOT been formally tested for perplexity.
They were compiled as an experiment to see how the model handles shifting bottlenecks. There is no guarantee that they are mathematically optimal or perform flawlessly.
If they pass the vibes test for you, enjoy!
.dat format for ik_llama.cpp compatibility.All variants use the same custom tensor buckets: attention, SSM, shared experts, routed experts, embeddings/output, and MTP/NextN tensors.
Highest quality routed expert quantization in this set.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | Q8_0 |
| Attention | Q8_0 |
| SSM Alpha & Beta | BF16 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts | IQ6_K |
| MTP / NextN | Q8_0 |
High quality routed expert quantization with IQ5_K experts.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | Q8_0 |
| Attention | Q8_0 |
| SSM Alpha & Beta | BF16 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts | IQ5_K |
| MTP / NextN | Q8_0 |
High quality routed expert quantization using IQ5_KS experts.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | Q8_0 |
| Attention | Q8_0 |
| SSM Alpha & Beta | BF16 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts | IQ5_KS |
| MTP / NextN | Q8_0 |
Balanced 4-bit routed expert quantization with high precision on always-active tensors.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | Q8_0 |
| Attention | Q8_0 |
| SSM Alpha & Beta | Q8_0 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts | IQ4_K |
| MTP / NextN | Q8_0 |
Smaller 4-bit routed expert quantization with compressed embeddings, output, and MTP tensors.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | IQ6_K |
| Attention | Q8_0 |
| SSM Alpha & Beta | Q8_0 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts | IQ4_KS |
| MTP / NextN | IQ6_K |
Ubergarm-style split routed expert recipe.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | IQ6_K |
| Attention | Q8_0 |
| SSM Alpha & Beta | Q8_0 |
| SSM Output | Q8_0 |
| Shared Experts | Q8_0 |
| Routed Experts Down | IQ4_KS |
| Routed Experts Gate/Up | IQ4_KSS |
| MTP / NextN | IQ6_K |
Lower size recipe with IQ3_K routed experts and IQ6_K on many always-active tensors.
| Layer Group | Quant |
|---|---|
| Token Embeddings & Output | IQ6_K |
| Attention | IQ6_K |
| SSM Alpha & Beta | Q8_0 |
| SSM Output | IQ6_K |
| Shared Experts | IQ6_K |
| Routed Experts | IQ3_K |
| MTP / NextN | IQ6_K |
Maximum compression variant in this set.
| Layer Group | Quant |
|---|---|
| Token Embeddings | IQ4_K |
| Output | IQ6_K |
| Attention | IQ6_K |
| SSM Alpha & Beta | IQ6_K |
| SSM Output | IQ6_K |
| Shared Experts | IQ6_K |
| Routed Experts Down | IQ3_KS |
| Routed Experts Gate/Up | IQ2_KL |
| MTP / NextN | IQ6_K |
ik_llama.cpp fork from ikawrakow/ik_llama.cpp.llama-server or llama-cli from that specific build.Example llama-server launch command:
./llama-server -m Qwopus3.5-122B-A10B-Kimi-K2.6-destill-healed-abliterated-IQ4_KS.gguf -c 8192 -ngl 99 -fa --jinja