Qwen3.5-122B-A10B-abliterated-FP8

Qwen3.5-122B-A10B-abliterated

Abliterated version of Qwen/Qwen3.5-122B-A10B with refusal direction removed.

Abliteration Details

• Method: Refusal direction projection removal (Arditi et al., 2024)
• Layers ablated: 5 (layers 43-47, covering both self_attn and linear_attn/Mamba layers)
• Tensors modified: 10 (o_proj/out_proj + q_proj/in_proj_qkv per layer)
• Alpha: 1.0 (full removal)
• Measurement: 64 harmful + 64 harmless prompts, strongest refusal signal at layer 48 (score: 78.4)

Architecture

• Type: Mixture of Experts (MoE) + Mamba hybrid attention
• Total params: 122B
• Active params: 10B per token (8/256 experts routed + 1 shared)
• Context: 262K tokens native
• Layers: 48 (13 self_attn + 36 linear_attn/Mamba)

Usage

Compatible with vLLM, transformers, and other inference frameworks that support Qwen3.5 MoE architecture.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Chompa1422/Qwen3.5-122B-A10B-abliterated",
    device_map="auto",
    trust_remote_code=True,
    dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained("Chompa1422/Qwen3.5-122B-A10B-abliterated")

Disclaimer

This model is intended for authorized security testing, CTF competitions, and educational purposes only.