Downloads · 30 days
164
4% of all-time downloads
wepiqx/Qwythos-9B-Claude-Mythos-5-1M-SHQ8-GGUF
Qwythos-9B-Claude-Mythos-5-1M-SHQ8-GGUF is a text generation model from wepiqx. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
Note on upstream v2: After these quants were made, the upstream release was updated to v2 with tokenizer normalization, embedded chat template improvements, and MTP variants. These quants are based on the original v1…
Downloads · 30 days
164
4% of all-time downloads
All-time downloads
3.8K
Public
Repo size
12.6 GB
Likes
7
Public
Click a slice to open those files.
.gguf12.6 GB · 100%
From the Hugging Face model README
Note on upstream v2: After these quants were made, the upstream release was updated to v2 with tokenizer normalization, embedded chat template improvements, and MTP variants. These quants are based on the original v1 source — they work fine with current llama.cpp for inference, but if you need the updated chat template or MTP support, grab the latest GGUFs from empero-ai/Qwythos-9B-Claude-Mythos-5-1M. MTP quants now available: wepiqx/Qwythos-9B-Claude-Mythos-5-1M-MTP-SHQ8-GGUF — same SHQ8 method with built-in MTP draft head for speculative decoding.
Note: File names contain
Q5_K_Mfor HF parser compatibility only. These are not pure Q5_K_M — they're selective hybrid quants using Q8_0, Q6_K, IQ4_XS, Q5_K, and F16 across different tensor types. See each section for the exact per-tensor map.
Selective hybrid quantizations for Empero's Qwythos-9B-Claude-Mythos-5-1M — a full-parameter reasoning fine-tune of Qwen3.5-9B with 1M context and vision support.
Uses the exact same SHQ8 method and formulas as Qwable-9B-Claude-Fable-5-SHQ8-GGUF — same architecture, same imatrix, same quantization strategies. This is not a new SHQ9 series; it's the proven SHQ8 approach applied to a different fine-tune.
| Quant | Size | PPL (ctx=1024) | vs Q6_K | vs OptA |
|---|---|---|---|---|
| Q6_K (baseline) | 7,008 MiB | 7.5876 ± 0.04948 | — | — |
| SHQ8-OptA ★ quality | 6,303 MiB | 7.4831 ± 0.04827 | −0.105, −705 MiB | — |
| SHQ8-v2 ★ compact | 5,726 MiB | 7.6542 ± 0.05003 | +0.067, −1,282 MiB | +0.171, −577 MiB |
Key finding: OptA formula transfers perfectly from Qwable (beats Q6_K PPL at 10% smaller). v2 shows a larger gap on Qwythos (+0.067) than on Qwable (+0.017) — IQ4_XS tensors impose a slightly higher cost on this fine-tune.
SHQ8-v2 vs stock quants: Despite the
Q5_K_Mname tag, SHQ8-v2 is actually better than a pure Q5_K_M — Q8_0 on critical attention paths + Q6_K on mid layers more than compensates for IQ4_XS on low-importance tensors. A stock Q5_K_M would be ~5.3 GB with PPL ~7.9 on Qwythos; v2 is 7.65 at nearly the same size. And vs Q3_K_M (~4.1 GB, PPL ~8.5+): v2 is only 1.6 GB larger but dramatically better quality. If you need the smallest possible file, consider v2 before dropping to low-bit quants.
Identical to Qwable-9B-Claude-Fable-5 — same Qwen3.5-9B backbone:
| Property | Value |
|---|---|
| Layers | 32 (24 Gated DeltaNet + 8 Full Attention) |
| Hidden dim | 4096 |
| FFN intermediate | 12288 |
| Vocabulary | 248,320 |
| Full Attention | blk.3, 7, 11, 15, 19, 23, 27, 31 |
| DeltaNet | all others |
| Context | 1,048,576 (YaRN factor 4.0) |
Tensors, shapes, and importance distribution are identical to Qwable. All SHQ8 techniques transfer directly.
Reused Qwable-9B-Claude-Fable-5.imatrix.gguf — same architecture, same tensor layout, same importance distribution. Confirmed working — OptA formula produces identical size (6,303 MiB) on Qwythos.
Q5_K_M base + Q8_0 on attn_gate/qkv/ssm_alpha/beta all layers + blk.31 ffn_down Q8_0 + F16 norms + imatrix per-row.
~/llm-tools/llama.cpp/build/bin/llama-quantize \
--imatrix /mnt/everything/qwen/source/Qwable-9B-Claude-Fable-5.imatrix.gguf \
--output-tensor-type Q5_K \
--token-embedding-type Q4_K \
--tensor-type "output_norm.*=Q8_0" \
--tensor-type "blk\.\d+\.attn_gate=Q8_0" \
--tensor-type "blk\.\d+\.attn_qkv=Q8_0" \
--tensor-type "blk\.\d+\.ssm_alpha=Q8_0" \
--tensor-type "blk\.\d+\.ssm_beta=Q8_0" \
--tensor-type "blk\.31\.ffn_down=Q8_0" \
--tensor-type ".*attn_q_norm.*=Q8_0" \
--tensor-type ".*attn_k_norm.*=Q8_0" \
--tensor-type ".*ssm_conv1d.*=Q8_0" \
--tensor-type ".*ssm_dt.*=F16" \
--tensor-type ".*ssm_a=F16" \
--tensor-type ".*attn_norm.*=F16" \
--tensor-type ".*post_attention_norm.*=F16" \
--tensor-type ".*ssm_norm.*=F16" \
/mnt/everything/qwen/source/Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf \
/mnt/everything/qwen/output/Qwythos-9B-Claude-Mythos-5-1M-SHQ8-OptA.gguf \
Q5_K_M
Q5_K_M base + tiered precision:
Config: configs/SHQ8_v2.sh (same as Qwable SHQ8-v2, adapted for Qwythos paths)
| Quant | Tokens/sec (GTX 1070) |
|---|---|
| SHQ8-OptA | ~26 t/s |
| SHQ8-v2 | ~28 t/s |
v2 is ~8% faster thanks to fewer Q8_0 tensors reducing VRAM bandwidth pressure on Pascal's limited PCIe bus.
Both quantizations generate full, working HTML/CSS/JS websites in a single pass at temperature 0.6 with the prompt:
"I'm a dev, my audience is youth. I like a creative/tech style. Write the full website code. This HTML will be our foundation."
A complete dev portfolio in 1072 lines:
Same prompt, more streamlined output in 580 lines:
At temp 0.6, both quants produce professional code. OptA goes for completeness (more sections, more features), v2 for conciseness. The reasoning fine-tune shows its quality — no repetition, no broken markup at this temperature.
Recommended sampling: Start with
temperature 0.6,top_k 20,top_p 0.95,min_p 0. If you encounter looping or over-thinking, setrepeat_penaltyto 1.05 — this solves both issues without touching temperature. Be cautious with high temperatures — this is a reasoning fine-tune and can get unstable above 1.2.Personal note from wepiqx: I've found that
top_p 1.0+min_p 0.05often produces noticeably better results thantop_p 0.95+min_p 0. The wider top_p pool with a minimum probability threshold gives the model more creative freedom while still filtering out low-probability garbage. Give it a try.
llama-cli \
-m M-SHQ8-OptA-Q5_K_M.gguf \
-p "Your prompt here" \
-ngl 99 \
--flash-attn on \
-c 4096 \
--temp 0.6 \
--top-k 20 \
--top-p 0.95 \
--min-p 0
For server mode (recommended):
llama-server \
-m M-SHQ8-OptA-Q5_K_M.gguf \
-c 65536 \
-fit off \
-ngl 99 \
--jinja --chat-template-file /path/to/chat_template.jinja \
--flash-attn on \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--port 8080 \
--mmap \
--temp 0.6 \
--top-p 0.95 \
--min-p 0 \
--top-k 20 \
--seed -1 \
--parallel 1
VRAM usage: ~7.4 GiB / 8 GiB at
-c 65536with flash-attn + q8_0 KV cache. The Gated DeltaNet architecture is memory-efficient even at long context.From wepiqx: This fine-tune can over-engineer at times, but overall it's a solid model. I'd love to see an MTP variant released — with MTP I could make an even better quantization with specialized draft model support for faster inference.
Modelfile:FROM ./M-SHQ8-OptA-Q5_K_M.gguf
TEMPLATE """{{ .System }}
{{ .Prompt }}"""
PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
PARAMETER min_p 0
ollama create qwythos-SHQ8 -f Modelfile
ollama run qwythos-SHQ8
If the model loops or over-thinks, set
repeat_penaltyto 1.05 — this solves both issues.
M-SHQ8-OptA-Q5_K_M.gguf into the apptemperature 0.6, top_k 20, top_p 0.95, min_p 0, repeat_penalty 1.0 (off)repeat_penalty to 1.05 — this solves both issues.Please be aware that Qwythos-9B inherits a deeply uncensored base and is fine-tuned to engage substantively with high-stakes technical domains like offensive cybersecurity, red-teaming methodologies, clinical medicine, and advanced pharmacology without refusals, hedging, or generic disclaimers.
| File | Size | Description |
|---|---|---|
Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf | 17 GB | BF16 source (from upstream) |
Qwythos-9B-Claude-Mythos-5-1M-Q6_K.gguf | 6.9 GB | Q6_K baseline |
M-SHQ8-OptA-Q5_K_M.gguf | 6.2 GB | Quality champion |
M-SHQ8-v2-Q5_K_M.gguf | 5.7 GB | Compact champion |