Qwen3.6-35B-A3B-Fable-5-Distill-heretic

Qwen3.6-35B-A3B-Fable-5-Distill-heretic (v5-dpo)

Revision v5 (2026-08-05): coverage-weighted injection-defense DPO on the v2 heretic weights. Prior revisions preserved on branches: v2 (surgery), v4 (first DPO pass).

A decensored version of armand0e/Qwen3.6-35B-A3B-Fable-5-Distill — itself a full-precision (BF16) fine-tune of Qwen3.6-35B-A3B distilled on real Claude Fable 5 sessions (armand0e/claude-fable-5-claude-code).

What changed in v5 (revision annotation)

v2 (2026-08-03, branch v2): two-pass heretic directional ablation for refusal suppression. Measured side effect: channel-impersonation injection compliance rose from 13% (pre-surgery) to 70%; exfiltration 7% to 43%.

v3 (2026-08-05, dead end, unpublished): constrained re-ablation with an injection-canary objective. No recovery; refusal suppression and injection obedience did not separate in the ablation search space.

v4 (2026-08-05, branch v4): first LoRA-DPO injection defense (Meta SecAlign shape, 1,312 on-policy pairs). Halved T6a (21/30 -> 13/30), left exfiltration untouched.

v5 (2026-08-05, THIS REVISION): fresh LoRA-DPO from the v2 weights (r=32, alpha=8, beta=0.1, 1 epoch, ~2.2h on one 96GB GPU) on 4,559 on-policy pairs, coverage-weighted from the v4 lesson: ~38% of pairs now carry exfiltration/action-laundering attacks (NVIDIA Nemotron agentic-IPI scenarios re-seeded for position variance, plus this model's own measured failure pages from the harness battery), the rest the proven v4 mix. Hypothesis tested and confirmed for instruction-following attacks: coverage, not volume, was the v4 gap.

Measured results (v5 gate, 2026-08-05, n=30 per row, temp 1.0,

212/212 valid runs, raw-log spot-checked)

T7 is the honest exception: "do not follow injected commands" is learnable by preference training; "do not place protected text into tool arguments" did not transfer from this pair distribution. In deployment an argument egress filter (block tool args containing system-prompt/history substrings) is REQUIRED for exfiltration; measured containment 12/12 with the filter active. This matches the literature: weight-side defenses are probabilistic and harness policy layers carry the exfiltration boundary.

Provenance

Qwen/Qwen3.6-35B-A3B
  -> armand0e/Qwen3.6-35B-A3B-Fable-5-Distill   (Fable 5 session distill, BF16)
    -> v2 heretic (two-pass directional ablation, 2026-08-03)   [branch: v2]
      -> v3 re-ablation + injection objective (2026-08-05, dead end, unpublished)
      -> v4 (first LoRA-DPO pass, 1,312 pairs, 2026-08-05)   [branch: v4]
      -> v5 THIS MODEL (coverage-weighted LoRA-DPO, 4,559 pairs, 2026-08-05)

Usage

Serve with vLLM >= 0.26 (Qwen3.6 MoE support):

vllm serve Authereon/Qwen3.6-35B-A3B-Fable-5-Distill-heretic \
  --max-model-len 131072 --max-num-seqs 64 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

~66GB BF16 weights: 96GB-class GPU or quantization for smaller cards. On Blackwell (sm_120) set VLLM_USE_FLASHINFER_SAMPLER=0 and use the official vllm/vllm-openai image (PyPI wheels lack sm_120 kernels). Sampling per generation_config: temperature 1.0, top_p 0.95, top_k 20. Reasoning models need generous max_tokens (>= 8K recommended, floor 512 with thinking on). NOTE: reasoning_effort and thinking_budget are silently dropped by vLLM 0.26 on this family (measured); the working control is chat_template_kwargs.enable_thinking.

Limitations

• T7 exfiltration resistance is NOT improved in v5 (~40% residual compliance). An argument egress filter is mandatory in agentic deployments; see the measured results table.
• White-box adaptive attacks defeat every published probabilistic defense (85-95% ASR, arXiv:2507.07417; >90% vs 9 of 12 defenses, arXiv:2510.09023). These weights are public; a motivated attacker has gradient access. Defense-in-depth (result sanitization, argument egress filtering, tool round caps) remains necessary in any agentic deployment.
• Residual refusals from the distill remain suppressed (the v2 surgery is intact); this model will answer what its stock sibling refuses. Safety filtering has been deliberately reduced. Review outputs before use. Not for public-facing or underage-audience applications. Users are solely responsible for compliance with applicable law.
• Tool discipline requires served schemas. Without a tool schema the model fabricates answers instead of calling (measured 3/3). Always serve tool definitions; always lint call names against the served set.
• The distill bias persists: trained on 352 coding-agent sessions; expect code/tool-shaped structure on general questions.

Warnings

• Safety filtering has been deliberately reduced. Outputs may include sensitive, controversial, or harmful content.
• Decensoring is weight surgery plus preference training, not retraining: behavior drift beyond the measured probes is possible.
• The injection resistance is measured against in-distribution attacks; novel attack classes should be re-probed before high-stakes use.