qwen3.5-9b-jspace-abliterated

Qwen3.5 9B - J-Space Abliterated

Qwen/Qwen3.5-9B with refusal behavior removed via J-space abliteration: a Jacobian-Lens-guided method that restricts the refusal direction to the causally-active workspace at each layer before ablation. Hybrid architecture-aware extraction and intervention are applied automatically to handle Qwen3.5's mix of full and linear attention layers.

Important: This model will produce uncensored outputs. Use responsibly.

Techniques Used

• J-Space Restricted Direction (Jacobian Lens): At each layer, J_ℓ = E[∂h_final/∂h_ℓ] is used to build a low-rank subspace spanned by data-mined refusal concept vectors (the causally-active "workspace" for refusal deliberation). The raw mean-difference refusal direction is projected onto this per-layer basis before being applied, so components that don't drive the emission of refusal tokens are dropped. Only the causally-clean part of the direction gets ablated.
• Data-Driven Concept Mining (Consensus): Refusal concepts are discovered empirically from the model's own next-token distribution on harmful prompts rather than hand-picked. Consensus mining walks the greedy continuation for 5 positions per prompt, scoring tokens by coverage × mean_prob. This picks up model-specific refusal openers a fixed anchor list would miss, and adapts to Qwen3.5's actual refusal vocabulary.
• Hybrid Architecture-Aware Intervention: Qwen3.5-9B interleaves full attention and linear attention layers (every 4th layer is full attention). Full attention layers receive full ablation weight (1.0x), while linear attention layers receive a reduced weight (0.4x). Recurrent dynamics projections (in_proj_a, in_proj_b) are skipped to preserve the delta-rule gating mechanism.
• Projected Refusal Direction (GrimJim's Method): The raw refusal direction is orthogonalized against the harmless mean to isolate the mechanistically-specific refusal component, avoiding damage to general helpfulness.
• Welford Mean + Float64 Subtraction: Numerically stable streaming mean computation and double-precision subtraction to handle high cosine similarity between harmful/harmless activation means.
• Norm Preservation: Maintains original Frobenius norms of weight matrices after projection.

Configuration

Architecture

Qwen3.5-9B is a hybrid attention vision-language model with 32 layers:

• 8 full attention layers (indices 3, 7, 11, 15, 19, 23, 27, 31)
• 24 linear attention layers (all others)
• Hidden size: 4096, attention heads: 16, KV heads: 4, head dim: 256
• Intermediate size: 12288, max context: 262144

How to read abliteration_config.json

The config records the J-space run in the shared J-space schema. Notable fields:

• j_space_mode: "restrict" — single-pass abliteration with per-layer direction restriction
• j_lens.min_projection_ratio: 0.1 — layers where the J-space basis retains <10% of the raw direction's norm fall back to the unrestricted direction (see logs for per-layer fell_back count)
• concepts.source: "auto-mine" with mining_mode: "consensus" — the concept list was derived from this model's own greedy continuations on the harmful prompt set

Credits

• Jacobian Lens / Workspace framing: Verbalizable Representations Form a Global Workspace in Language Models (transformer-circuits.pub, 2026)
• Norm-Preserving Biprojected Abliteration — Jim Lai (grimjim) (2025)
• Base Model: Qwen/Qwen3.5-9B by the Qwen Team

Toolkit

github.com/jwest33/abliterator

License

This model inherits the Apache 2.0 license from the base model.

Disclaimer

This model is provided for research and educational purposes. The creators are not responsible for any misuse. Users are solely responsible for ensuring their use complies with applicable laws and ethical standards.