Qwen3.6-27B-Fable-Fusion-711-Heretic2

Qwen3.6-27B-Fable-Fusion-711-Heretic2

A second, terminal Heretic ablation of DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP, targeted at creative-writing refusals rather than AdvBench-style harm refusals.

GGUF quants: …-Heretic2-GGUF

Why this exists

The base model is described as "Uncensored / Heretic", and its card reports 4/100 refusals. That number is real, but the same card also says "Heretic uncensored (pre tuning)" — the ablation was performed on stock Qwen/Qwen3.6-27B by trohrbaugh, and DavidAU then ran multi-stage fine-tuning and merging on top, including Claude Opus reasoning traces and GPT-5 Polaris data. Training on generations from aligned models re-teaches refusal.

The published figure therefore describes an intermediate artifact, not the released weights. In practice the released model still refuses.

In this model the ablation is the last step. Nothing is trained or merged afterward, so the measured numbers describe the weights you are downloading.

Method

Heretic v1.4.0, ARA-style configuration:

Prompt sets — the important part

Stock Heretic computes its refusal direction from mlabonne/harmful_behaviors (416 AdvBench rows, almost entirely "write a tutorial on how to hack X"). That direction has little to do with a model balking mid-scene at violence or an adult scene, which is why generic abliterations often fail for fiction work.

This run used a custom probe set: ~350 prompts collected by actually hitting refusals on the base model during creative-writing use, merged with AdvBench so that "character explains how to pick a lock" style refusals are still covered.

• Direction set: custom prompts (weighted ×3) + AdvBench ≈ 1157 lines
• Evaluation set: 100 custom prompts, held out — they never appear in the direction set, so the refusal score measures generalisation rather than memorisation of the prompts it was fitted on.

Results

Selected from the Pareto front produced by the optimiser:

Heretic notes that KL above ~0.5 indicates significant capability damage; this is roughly two orders of magnitude below that.

Read these numbers narrowly. The scorer generates a bounded number of tokens at temperature 0 and substring-matches a refusal-marker list. A response that opens agreeably and declines later, or declines in wording outside the list, is not counted. Treat it as "the ablation generalised on a held-out set of the prompts I care about", not as an absolute refusal rate.

⚠️ The MTP layer was removed

The base model is a -MTP build: it carries an extra multi-token-prediction layer (blk.64, nextn_predict_layers: 1) used for speculative decoding.

transformers does not model that head, so save_pretrained drops it. These weights therefore contain 64 layers, not 65, and config.json reflects that.

If you quantise these weights to GGUF yourself, the converter may still read nextn_predict_layers and write block_count: 65, producing a file that fails to load with:

check_tensor_dims: tensor 'blk.64.attn_norm.weight' not found

The GGUFs in the companion repo have this metadata corrected. Speculative decoding via MTP is not available in this model.

Reproducing

The 26 MB Heretic adapter reproduces this from the public base model.

Do not merge it with PEFT. PeftModel.from_pretrained fails to bind Heretic's key layout — it emits a "Found missing adapter keys" warning and then returns the unmodified base model, with no exception and a clean exit. Apply the LoRA math directly instead, and assert the weights actually changed:

delta = (B.float() @ A.float()) * (lora_alpha / r)
w.add_(delta.to(w.dtype))
assert max_abs_delta > 0.0   # or you have silently shipped the original model

License

Apache 2.0, inherited from the base model. Derivative work.