Downloads · 30 days
31
7% of all-time downloads
Qwythos-9B-PerfectSplit-CoT-GGUF
Aronstk15/Qwythos-9B-PerfectSplit-CoT-GGUF
Qwythos-9B-PerfectSplit-CoT-GGUF is a text generation model from Aronstk15. Use it when you need the model to write or continue text. It is set up for llama-cpp-python. The card lists the license as apache-2.0.
A custom mixed-precision GGUF quantization of Qwythos-9B-Claude-Mythos-5-1M, surgically optimized to preserve Chain-of-Thought (CoT) reasoning quality and 1M-token context recall while reducing the model size by ~37%.
All-time downloads
473
Public
Repo size
23.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf11.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- llama-cpp-python
- License
- apache-2.0
- Access
- Public
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026
- SHA
- 34f82366
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwythos-9B-PerfectSplit-CoT-GGUF
A custom mixed-precision GGUF quantization of Qwythos-9B-Claude-Mythos-5-1M, surgically optimized to preserve Chain-of-Thought (CoT) reasoning quality and 1M-token context recall while reducing the model size by ~37%.
🎯 Why This Quantization Exists
Standard uniform quantizations (Q4_K_M, Q5_K_M, etc.) apply the same bit-width to every tensor. But in reasoning models with hybrid attention stacks, not all weights are equal:
- Attention & DeltaNet gates control information routing and memory retention. Even 1-bit of error here causes attention score drift at long context lengths and breaks the linear recurrence that powers the 1M context window.
- MLP layers contain massive mathematical redundancy due to their expansion ratio. They tolerate aggressive quantization with virtually no quality loss.
This model exploits that asymmetry by keeping sensitive layers at full F16 precision and only compressing the redundant MLP layers.
🧬 Quantization Strategy: "Perfect Split"
| Component | Precision | Rationale |
|---|---|---|
Token Embeddings (token_embd.weight) | F16 | First point of contact with input. Errors propagate through every layer. |
Output Head (output.weight / lm_head) | F16 | Maps hidden states to vocabulary logits. Quantization here causes wrong token selection and repetition loops. |
Attention Projections (attn_q, attn_k, attn_v, attn_gate, attn_qkv) | F16 | Prevents attention score drift at 128K+ context lengths. |
DeltaNet / SSM (ssm_a, ssm_alpha, ssm_beta, ssm_dt, ssm_conv1d, ssm_out, ssm_norm) | F16 | The recurrence S_t = α_t ⊙ S_{t-1} + β_t ⊙ (k_t ⊗ v_t) runs for millions of steps. Any error accumulates multiplicatively. |
Norms (attn_norm, ffn_norm, ssm_norm) | F16 | Prevents activation scaling drift across 32+ layers. |
MLP Layers (ffn_gate, ffn_up, ffn_down) | Q5_K_M | Highly redundant due to expansion ratio. Safe to compress. |
Final Statistics
| Metric | Value |
|---|---|
| Original Size (F16) | 18.00 GB |
| Final Size | 10.88 GB |
| Compression | ~37% smaller |
| Average BPW | 10.43 |
| F16 Tensors | 331 |
| Q5_K_M Tensors | 96 |
| Total Tensors | 427 |
📊 Expected Quality vs. Standard Quants
| Metric | Standard Q5_K_M | PerfectSplit-CoT | Full BF16 |
|---|---|---|---|
| CoT Chain Coherence (50+ steps) | Good | Excellent | Excellent |
| Repetition Loops | Low risk | Minimal | Minimal |
| Long Context Recall (256K+) | Moderate drift | Excellent | Excellent |
| Tool Call JSON Accuracy | ~92% | ~97%+ | ~98% |
| Factual Knowledge Recall | Very Good | Very Good | Excellent |
| File Size | ~6.5 GB | 10.88 GB | 18.00 GB |
Trade-off: This model is ~4 GB larger than a standard Q5_K_M, but preserves near-BF16 quality for reasoning and long-context tasks.
🚀 Usage
With llama.cpp
./llama-cli \
-m Qwythos-9B-PerfectSplit-CoT.gguf \
--prompt "Analyze this dataset and explain the trends." \
--n-predict 2000 \
--temp 0.6 \
--top-k 20 \
--top-p 0.95 \
--min-p 0.05 \
--repeat-penalty 1.05 \
--ctx-size 32768 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--flash-attn \
--ngl 99
- Task
- Text Generation
- Library
- llama-cpp-python
- License
- apache-2.0
- Created
- Jul 22, 2026
- Updated
- Jul 23, 2026