BigBang-Aquila-35B-Merged-Abliterated

BigBang-Aquila-35B-Merged-Abliterated

Abliterated (OrthoBot) version of the merged model osk-arr00/BigBang-Aquila-35B-Merged.

What it is

DARE-TIES merge of BigBang-v1 + XYZ-Aquila-mini (base Qwen/Qwen3.6-35B-A3B), with the refusal direction orthogonalized from the weights that write to the residual stream.

• Method: OrthoBot (forward-only, 64 harmful + 64 harmless prompts)
• Tensors ablated: 50 (attention o_proj + expert/shared down_proj)
• refusal_dirs: significant (mean 1.89, min 0.14, max 27.9) — not noise
• Verified: cosine original vs abliterated < 1.0 on ablated modules

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("osk-arr00/BigBang-Aquila-35B-Merged-Abliterated")
tokenizer = AutoTokenizer.from_pretrained("osk-arr00/BigBang-Aquila-35B-Merged-Abliterated")

Requires transformers>=5.12 + kernels flash-linear-attention==0.4.2 and causal_conv1d==1.6.0 (qwen3_5_moe architecture with Gated DeltaNet).

Derivatives

The quantized GGUFs (APEX-IQ, Q4_K_M) were generated from this checkpoint: → osk-arr00/BigBang-Aquila-35B-GGUF

Notes

• Text-only checkpoint (Qwen3_5MoeForCausalLM). The vision tower was dropped during abliteration (loaded via the text path).
• MTP: this checkpoint does NOT include MTP weights (abliterate.py does not save the MTP head). To use MTP, re-insert the 19 MTP tensors from Qwen/Qwen3.6-35B-A3B (see insert_mtp.py).