Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic

<div style="background-color: #ff4444; color: white; padding: 20px; border-radius: 10px; text-align: center; margin: 20px 0;"> <h2 style="color: white; margin: 0 0 10px 0;">🚨⚠️ I HAVE REACHED HUGGING FACE'S FREE STORAGE LIMIT ⚠️🚨</h2> <p style="font-size: 18px; margin: 0 0 15px 0;">I can no longer upload new models unless I can cover the cost of additional storage.<br>I host <b>70+ free models</b> as an independent contributor and this work is unpaid.<br><b>Without your support, no more new models can be uploaded.</b></p> <p style="font-size: 20px; margin: 0;"> <a href="https://patreon.com/LLMfan46" style="color: white; text-decoration: underline;">🎉 Patreon (Monthly)</a> &nbsp;|&nbsp; <a href="https://ko-fi.com/llmfan46" style="color: white; text-decoration: underline;">☕ Ko-fi (One-time)</a> </p> <p style="font-size: 16px; margin: 10px 0 0 0;">Every contribution goes directly toward Hugging Face storage fees to keep models free for everyone.</p> </div>

---

85% fewer refusals (11/100 Uncensored vs 73/100 Original) while preserving model quality (0.0123 KL divergence).

❤️ Support My Work

Creating these models takes significant time, work and compute. If you find them useful consider supporting me:

Your help will motivate me and would go into further improving my workflow and coverings fees for storage, compute and may even help uncensoring bigger model with rental Cloud GPUs.

---

This is a decensored version of a empero-ai/Qwythos-9B-Claude-Mythos-5-1M, made using Heretic v1.2.0 with a variant of the Magnitude-Preserving Orthogonal Ablation (MPOA) method

Abliteration parameters

Targeted components

• attn.o_proj
• attn.out_proj
• mlp.down_proj

Performance

MMLU test results:

<span style="color:blue">Original:</span>

============================================================

• Total questions: 7021

• Correct: 5408

• Accuracy: 0.7703 (77.03%)

• Parse failures: 0

============================================================

Tested subject scores:

• professional_law: 0.5885 (462/785)
• moral_scenarios: 0.5339 (236/442)
• miscellaneous: 0.8851 (339/383)
• professional_psychology: 0.8259 (261/316)
• high_school_psychology: 0.9593 (259/270)
• high_school_macroeconomics: 0.8477 (167/197)
• elementary_mathematics: 0.7011 (129/184)
• moral_disputes: 0.8046 (140/174)
• prehistory: 0.8372 (144/172)
• philosophy: 0.7673 (122/159)
• high_school_biology: 0.9276 (141/152)
• professional_accounting: 0.6713 (96/143)
• clinical_knowledge: 0.8286 (116/140)
• high_school_microeconomics: 0.9485 (129/136)
• nutrition: 0.8296 (112/135)
• professional_medicine: 0.8582 (115/134)
• conceptual_physics: 0.8359 (107/128)
• high_school_mathematics: 0.5591 (71/127)
• human_aging: 0.7586 (88/116)
• security_studies: 0.7857 (88/112)
• high_school_statistics: 0.7748 (86/111)
• marketing: 0.9450 (103/109)
• high_school_world_history: 0.9057 (96/106)
• sociology: 0.8932 (92/103)
• high_school_government_and_politics: 0.9505 (96/101)
• high_school_geography: 0.9293 (92/99)
• high_school_chemistry: 0.7526 (73/97)
• high_school_us_history: 0.9158 (87/95)
• virology: 0.5281 (47/89)
• college_medicine: 0.8295 (73/88)
• world_religions: 0.8409 (74/88)
• high_school_physics: 0.6429 (54/84)
• electrical_engineering: 0.7654 (62/81)
• astronomy: 0.9494 (75/79)
• logical_fallacies: 0.8553 (65/76)
• high_school_european_history: 0.9041 (66/73)
• anatomy: 0.7887 (56/71)
• college_biology: 0.9375 (60/64)
• human_sexuality: 0.8125 (52/64)
• formal_logic: 0.6719 (43/64)
• public_relations: 0.7049 (43/61)
• international_law: 0.9000 (54/60)
• college_physics: 0.6842 (39/57)
• college_mathematics: 0.5455 (30/55)
• econometrics: 0.6296 (34/54)
• jurisprudence: 0.8491 (45/53)
• high_school_computer_science: 0.8654 (45/52)
• machine_learning: 0.6154 (32/52)
• medical_genetics: 0.8627 (44/51)
• global_facts: 0.3725 (19/51)
• management: 0.9200 (46/50)
• us_foreign_policy: 0.9400 (47/50)
• college_chemistry: 0.5532 (26/47)
• abstract_algebra: 0.5957 (28/47)
• business_ethics: 0.6957 (32/46)
• college_computer_science: 0.8000 (36/45)
• computer_security: 0.7907 (34/43)

<span style="color:darkgreen">Heretic:</span>

============================================================

• Total questions: 7021

• Correct: 5431

• Accuracy: 0.7735 (77.35%)

• Parse failures: 0

============================================================

Tested subject scores:

• professional_law: 0.6000 (471/785)
• moral_scenarios: 0.5294 (234/442)
• miscellaneous: 0.8930 (342/383)
• professional_psychology: 0.8259 (261/316)
• high_school_psychology: 0.9630 (260/270)
• high_school_macroeconomics: 0.8426 (166/197)
• elementary_mathematics: 0.7011 (129/184)
• moral_disputes: 0.8161 (142/174)
• prehistory: 0.8430 (145/172)
• philosophy: 0.7673 (122/159)
• high_school_biology: 0.9276 (141/152)
• professional_accounting: 0.6783 (97/143)
• clinical_knowledge: 0.8286 (116/140)
• high_school_microeconomics: 0.9559 (130/136)
• nutrition: 0.8370 (113/135)
• professional_medicine: 0.8657 (116/134)
• conceptual_physics: 0.8359 (107/128)
• high_school_mathematics: 0.5748 (73/127)
• human_aging: 0.7586 (88/116)
• security_studies: 0.7857 (88/112)
• high_school_statistics: 0.7838 (87/111)
• marketing: 0.9358 (102/109)
• high_school_world_history: 0.9057 (96/106)
• sociology: 0.8932 (92/103)
• high_school_government_and_politics: 0.9406 (95/101)
• high_school_geography: 0.9293 (92/99)
• high_school_chemistry: 0.7526 (73/97)
• high_school_us_history: 0.9053 (86/95)
• virology: 0.5281 (47/89)
• college_medicine: 0.8295 (73/88)
• world_religions: 0.8523 (75/88)
• high_school_physics: 0.6429 (54/84)
• electrical_engineering: 0.7654 (62/81)
• astronomy: 0.9367 (74/79)
• logical_fallacies: 0.8553 (65/76)
• high_school_european_history: 0.9178 (67/73)
• anatomy: 0.7887 (56/71)
• college_biology: 0.9531 (61/64)
• human_sexuality: 0.7969 (51/64)
• formal_logic: 0.6562 (42/64)
• public_relations: 0.7049 (43/61)
• international_law: 0.9000 (54/60)
• college_physics: 0.6491 (37/57)
• college_mathematics: 0.5636 (31/55)
• econometrics: 0.6296 (34/54)
• jurisprudence: 0.8491 (45/53)
• high_school_computer_science: 0.8654 (45/52)
• machine_learning: 0.6346 (33/52)
• medical_genetics: 0.8627 (44/51)
• global_facts: 0.4314 (22/51)
• management: 0.9200 (46/50)
• us_foreign_policy: 0.9400 (47/50)
• college_chemistry: 0.5319 (25/47)
• abstract_algebra: 0.6170 (29/47)
• business_ethics: 0.7174 (33/46)
• college_computer_science: 0.8222 (37/45)
• computer_security: 0.8140 (35/43)

MMLU - Massive Multitask Language Understanding, multiple-choice questions across 57 subjects (math, history, law, medicine, etc.).

GGUF Version

GGUF quantizations available here llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF.

---

<p align="center"> <img src="assets/qwythos.png" alt="Qwythos-9B" width="640"/> </p>

Qwythos-9B

Developed by Empero

Qwythos-9B is a full-parameter reasoning model built on top of a deeply uncensored Qwen3.5-9B base and post-trained on over 500 million tokens of high-quality Claude Mythos and Claude Fable traces, with chain-of-thought generated in-house by Empero AI's internal tool rethink.

The result is a compact, fast, dramatically more capable 9B reasoning model. Headline capabilities:

• 🔭 1,048,576-token context — Qwythos ships with YaRN rope-scaling enabled by default for a full 1M-token context window out of the box. One of the longest context windows available in any 9B-class open-weight model, suitable for whole-codebase reasoning, multi-document research, and long agentic trajectories.
• 📈 Dominates the base under matched evaluation: +34 pts MMLU, +30 pts gsm8k-strict, +19 pts gsm8k-flex.
• 🛠 Native function calling per Qwen3.5's spec — no extra wrapper, no tool-specific fine-tune required.
• 🎯 Self-corrects with tools — when given a Python executor and a web search tool, Qwythos produced source-cited, factually-correct answers on 7 of 7 test prompts spanning math, cybersecurity, clinical pharmacology, and biochemistry.

Qwythos is intentionally uncensored. It is designed to engage seriously with technically demanding questions across cybersecurity, red-teaming methodology, biology, pharmacology, and clinical medicine — domains where over-aligned models tend to refuse, hedge into uselessness, or surface boilerplate disclaimers in place of substance.

---

Headline results

<p align="center"> <img src="assets/qwythos_eval_chart.svg" alt="Qwythos vs. base Qwen3.5-9B across seven benchmarks" width="900"/> </p>

Same harness. Same sampling. Same prompts. The wins are real.

All numbers produced with lm-evaluation-harness, HF backend, --apply_chat_template, Qwen3.5 sampling (temperature=0.6, top_p=0.95, top_k=20), --limit 100. Full per-task and per-subject (MMLU) breakdown in evals/lm_eval_results.md. Raw results*.json and per-sample samples_*.jsonl are available on request.

The MMLU +34.3 lift is the headline. Qwythos posts 0.575 mean across all 57 subjects, peaking at 0.78 on government/politics, 0.77 on college biology, 0.74 on conceptual physics — placing it well above what most 9B reasoning models deliver under the same evaluation conditions. Absolute MMLU numbers for any 9B model are sensitive to harness, few-shot count, and chat-template handling; what matters in this comparison is that both models were evaluated with identical settings.

---

Capability: Native tool use with self-correction

Qwythos supports OpenAI/Qwen3.5-style function calling out of the box — no extra wrapper, no fine-tune-on-tools needed. Pass tools=[...] to the chat template and the model emits valid <tool_call> blocks per Qwen3.5's spec, with required parameters honored.

We evaluated tool use on a 7-prompt harness combining capability demos with deliberately hard factual-recall prompts where closed-book sampling fails:

7 of 7 succeeded. Tool selection was always sensible (math → Python; facts → search). The four bottom rows are particularly important: they are the four hardest specialty facts to recall closed-book — and Qwythos, given the right tools, searched, integrated multiple sources, and produced source-cited correct answers in every case.

Full transcripts with the model's reasoning, every tool call issued, every result returned, and the final integrated answer are in evals/tool_test_outputs.md.

This makes Qwythos deployment-ready for retrieval-augmented agentic settings, where the model verifies its specifics rather than fabricating them.

---

Capability: 1,048,576-token context window

Qwythos ships with YaRN rope-scaling configured by default for a 1,048,576-token (≈1M) context window — a 4× extension over the 262,144-token native architecture. The configuration is baked into config.json and applies automatically at load time; no separate flag, post-processing step, or YaRN-specific tokenizer is required:

"rope_parameters": {
  "rope_type": "yarn",
  "factor": 4.0,
  "original_max_position_embeddings": 262144,
  "mrope_interleaved": true,
  "mrope_section": [11, 11, 10],
  "rope_theta": 10000000
},
"max_position_embeddings": 1048576

This is the official Qwen3.5 recipe for 1M context, matching the configuration documented in Qwen's own model card and the vLLM/SGLang deployment recipes. Long-context inference was validated on this checkpoint via in-house smoke testing at ~137k tokens.

What 1M context unlocks:

• Whole-codebase reasoning. A 1M-token window comfortably fits multi-hundred-thousand-line repositories — enabling cross-file refactoring, defect-finding, and architectural review without RAG chunking.
• Long agentic trajectories. Multi-round tool-use sessions with verbose tool outputs (large web-search hit sets, paginated API responses, long Python tracebacks) stay in-context across dozens of turns.
• Multi-document research. A typical research session (10–20 papers + notes + the user's working draft) fits in one prompt — synthesize across all of them in a single forward pass.
• Long-form scientific reasoning. Chains of <think> reasoning over multi-paper biomedical or pharmacological corpora.

Serving at 1M:

# vLLM
vllm serve empero-ai/Qwythos-9B-Claude-Mythos-5-1M --max-model-len 1010000

# SGLang
SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 python -m sglang.launch_server \
  --model-path empero-ai/Qwythos-9B-Claude-Mythos-5-1M --context-length 1010000

Practical notes:

• The full 1M window benefits from tensor-parallel multi-GPU or aggressive KV-cache offload — a single H100/H200 comfortably handles 256k–512k. Below ~256k tokens of context, the hybrid Gated-DeltaNet attention stack keeps memory growth sub-quadratic, so long contexts are dramatically cheaper than they'd be on a pure full-attention model of similar size.
• Static YaRN at factor=4.0 introduces a small short-context quality cost (a known YaRN trade-off across the industry). For workloads that never exceed the native 262k window and want maximum short-context fidelity, restore rope_parameters.rope_type to "default" from the included config.json.pre_yarn backup.

Reproducing the tool harness

The harness is a small ~150-line Python file:

• python_executor(code) — runs Python in a subprocess (12s timeout, captured stdout/stderr)
• web_search(query, max_results) — DuckDuckGo via the ddgs package

Pass both as tools= to apply_chat_template and parse <tool_call> blocks from the model's output. The parser handles Qwen3.5's chat-template format:

<tool_call>
<function=NAME>
  <parameter=PARAM>value</parameter>
</function>
</tool_call>

Empero will release the reference harness on GitHub.

---

Sampling recommendations

Qwythos was trained as a reasoning model and inherits Qwen3.5's thinking-mode behavior. Use these settings as defaults:

gen_kwargs = dict(
    do_sample=True,
    temperature=0.6,    # Qwen3.5 thinking-mode recommended
    top_p=0.95,
    top_k=20,
    repetition_penalty=1.05,
    max_new_tokens=16384,  # generous budget for the <think> reasoning block + final answer
)

Why these: in a controlled retest (see evals/retest_outputs.md), we evaluated multiple sampling configurations against the three most-difficult factual prompts. Greedy decoding and very-low-temperature sampling (T≤0.3) degenerated into repetition loops — a known failure mode for reasoning models on this class of prompts. Qwen3.5's recommended setting (T=0.6) cleanly avoids this and delivers the best factual reliability we measured: across the three retest prompts, zero of the six errors flagged in closed-book review recurred at T=0.6 — including the safety-relevant physostigmine claim, the misattributed CVE, and the incorrect hashcat hash-mode.

Use repetition_penalty=1.05 — a small deviation from Qwen's default of 1.0 that prevents rare non-terminating reasoning loops on long generations.

---

Domain coverage

Qwythos is a general-purpose reasoning model with explicit emphasis on cybersecurity, biomedical, and quantitative reasoning. From the qualitative sample-generations review across 25 prompts spanning these domains (full transcripts in evals/sample_generations.md):

• Cybersecurity — produces detailed defender-oriented walkthroughs of SQL injection mitigations, TLS handshake structure, EDR/process-injection detection, Linux hardening, MITRE ATT&CK ransomware kill chains.
• Red-team methodology — clean explanations of engagement phases, scoping, rules of engagement, evidence handling, reporting. Especially strong on social-engineering pretext analysis and phishing-resistant defenses.
• Biology / biochemistry — step-by-step mechanisms for CRISPR-Cas9, mRNA vaccines, SARS-CoV-2 spike protein, antibiotic-resistance mechanisms, organophosphate AChE inhibition.
• Pharmacology — strong on receptor pharmacology fundamentals (agonism, antagonism, partial agonism with worked examples), statin mechanism, opioid respiratory depression at the brainstem level, beta-blocker indications, therapeutic-window reasoning for narrow-index drugs.
• Clinical medicine — ACS chest-pain differential and workup, type-2 diabetes pathophysiology and drug-class targeting, sepsis recognition (qSOFA) and bundle.
• Math — strong at gsm8k-style multi-step word problems, minerva-style competition math; 86% gsm8k, integer arithmetic verified by python_executor when invoked.

The uncensored base means Qwythos engages substantively with these prompts rather than refusing, hedging, or burying answers in disclaimer boilerplate. Reasoning is shown in the <think> block; final answer follows.

---

Model details

• Base model: Qwen/Qwen3.5-9B — a dense, natively multimodal architecture with a hybrid attention stack (3:1 Gated DeltaNet linear-attention to Gated full-attention), ~152k vocabulary, long native context.
• Fine-tune type: full parameter (all text-backbone weights trained). The vision tower was frozen — training was text-only, so vision behavior is inherited from the base and was not tuned or tested.
• Objective: supervised fine-tuning, assistant-only loss (the model is scored only on the assistant/completion tokens; prompts are masked).
• Context length: 1,048,576 tokens (≈1M) — YaRN rope-scaling enabled by default in config.json. Native architectural context is 262,144 tokens; YaRN factor 4.0 extends this to the full 1M window without any retraining or runtime flag, matching Qwen's official long-context recipe.
• License: Apache 2.0.

Training data

Qwythos was post-trained on over 500 million tokens of high-quality reasoning data drawn from:

• Claude Mythos and Claude Fable traces — long, multi-turn problem-solving conversations spanning code, math, science reasoning, biomedical analysis, and agentic tool use.
• Chain-of-thought generated in-house by rethink, Empero AI's internal CoT-generation tool. rethink produces deliberately structured <think>-block reasoning that walks through hypothesis, verification, and conclusion before the final answer is committed — directly shaping Qwythos's reason-then-answer behavior.

All data was normalized to Qwen3.5's chat format. Training used assistant-only loss so the model is scored only on completion tokens.

Training procedure

Full-parameter supervised fine-tuning with TRL:

Held-out validation loss decreased monotonically across both phases (final eval_loss ≈ 0.709, mean token accuracy 0.799 on a curated holdout). No overfitting observed.

---

How to use

The base is multimodal; for text-only inference load with AutoModelForImageTextToText:

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "empero-ai/Qwythos-9B-Claude-Mythos-5-1M"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype="bfloat16", device_map="auto"
)

messages = [
    {"role": "user",
     "content": "Walk through the biochemistry of how organophosphate nerve agents inhibit acetylcholinesterase, the resulting cholinergic toxicity, and the medical antidotes."}
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)

out = model.generate(
    **inputs, max_new_tokens=16384, do_sample=True,
    temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05,
)
# Output opens with <think>...</think> reasoning, then the final answer.
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With tools (function calling)

TOOLS = [
    {"type": "function", "function": {
        "name": "python_executor",
        "description": "Execute Python code and return stdout.",
        "parameters": {"type": "object",
                       "properties": {"code": {"type": "string"}},
                       "required": ["code"]}}},
    {"type": "function", "function": {
        "name": "web_search",
        "description": "Search the web for current facts and citations.",
        "parameters": {"type": "object",
                       "properties": {"query": {"type": "string"},
                                      "max_results": {"type": "integer"}},
                       "required": ["query"]}}},
]

text = tok.apply_chat_template(messages, tools=TOOLS, tokenize=False, add_generation_prompt=True)
# ... then parse <tool_call><function=...><parameter=...>...</parameter></function></tool_call> blocks

Requirements: a recent transformers (Qwen3.5 support) plus the Gated DeltaNet kernels (flash-linear-attention and a CUDA-matched causal_conv1d build) — without them the linear-attention layers fall back to slow, memory-hungry PyTorch ops.

---

Limitations

Qwythos is a focused 9B reasoning model. A few characteristics are worth knowing to get the best out of it:

• It's a reasoning model. Every answer opens with a <think> block before the final response. Allow generous max_new_tokens (16,384 recommended) and parse/strip the <think>...</think> span for end users.
• Use recommended sampling. At greedy decoding or very-low-temperature (T≤0.3) sampling, the model can enter repetition loops on long generations — a known reasoning-model failure mode. Use temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05 for consistently crisp results.
• Verify specifics in safety-critical contexts. Like all closed-book LLMs in this weight class, Qwythos can over-commit to specific identifiers (CVEs, hashcat modes, exact biochem positions, drug-label numerics) it isn't certain about. The tool-augmented path (Python executor + web search) cleanly resolves this in our evaluation — for deployments where exact identifiers matter, pair Qwythos with retrieval or function calling.
• Uncensored. Qwythos inherits a deeply uncensored base and does not refuse or hedge on technically demanding questions. Add your own application-level review/safety layer for end-user-facing deployments where that matters.
• Text-only fine-tune. The base is multimodal, but only the text path was trained. Vision behavior is inherited from the base and was not evaluated here.

---

Stay in the loop

Sign up for the Empero newsletter at empero.org for releases, evals, and research notes on Qwythos and future open-weight models from the lab.

Support / Donate

If this model helped you, consider supporting the project:

• BTC: bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v
• LTC: ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x
• XMR: 42Dbm5xg5Nq26fdyzfEU7KBnAJfhi7Cvz5J2ex5CzHXkfKuNEJzYCcmJ1GTbgjFZ5MBx72sdG1G9239Cd6rsZfv4QeDkYJY

---

Provenance & licensing

Weights are released under Apache-2.0, inherited from the Qwen3.5-9B base. Shared for research and experimentation, as-is.

Acknowledgements

• Developed and released by Empero
• Base model: Qwen3.5-9B (Alibaba Qwen team)
• Training: TRL + Transformers
• Linear-attention kernels: flash-linear-attention, causal_conv1d
• Evaluation: lm-evaluation-harness (EleutherAI)