Qwen3-VL-235B-A22B-Abliterated-FP8

Qwen3-VL-235B-A22B-Abliterated (FP8)

WARNING: This model has had its safety alignment removed. It may generate content that is offensive, harmful, or illegal. Not suitable for all audiences. Use at your own risk.

This is a version of Qwen/Qwen3-VL-235B-A22B with refusal behavior removed via SVD multi-direction abliteration — a technique that extends standard abliteration to work on vision-language MoE models where the single-direction approach fails.

Quantized to FP8 (float8_e4m3) for efficient deployment. 222 GB on disk, servable via vLLM.

Why SVD?

Standard abliteration computes one refusal direction per layer and projects it out. This works on text-only models but fails on VL models — the mean refusal signal is ~11x weaker (signal quality 0.01 vs 0.12 on text-only Qwen3-235B). Seven iterations of standard abliteration plateaued at ~60% refusal rate.

SVD analysis revealed the cause: the VL model encodes refusal along multiple orthogonal directions that cancel when averaged. 93 of 94 layers have a top-1 SVD direction explaining less than 80% of refusal variance. By extracting and projecting out the top-k directions simultaneously, we capture the full refusal subspace.

Evaluation

Tested on 100 adversarial prompts across 8 categories + 50 benign prompts:

Per-Category Breakdown (Retry Mode)

Intended Uses

• Research: Studying representation engineering, safety alignment mechanisms, and refusal behavior in large language models
• Red-teaming: Evaluating model vulnerabilities in controlled environments
• Creative writing: Fiction and narrative generation in controlled, private settings with appropriate human oversight

Out-of-Scope Uses

• Production or public-facing applications without human oversight and content filtering
• Applications targeting minors or vulnerable populations
• Generation of content that violates applicable laws or regulations
• Circumventing safety controls in regulated domains (healthcare, finance, legal)
• Any use that violates the Hugging Face Content Policy

How It Works

The abliteration technique (originally by FailSpy, popularized by mlabonne) identifies refusal directions in a model's activation space and projects them out of weight matrices. This model extends the technique with SVD:

1. Collect per-sample activations — Run adversarial prompts through the model, store individual hidden states per layer (not just the mean)
2. Compute SVD — Per layer, form D = harmful_activations - harmless_mean and compute truncated SVD to extract the top-k orthogonal refusal directions
3. Project out a subspace — Remove a rank-k subspace from o_proj and down_proj weight matrices (including all MoE expert weights), weighting each direction by its singular value

Configuration

• SVD rank: 32 directions measured, top 8 used for ablation
• SV weighting: sv[i] / sv[0] — strongest direction at full scale, weaker proportionally
• Layers ablated: 74 (layers 20–93), dual-peak scale curve at layers 56 and 71
• Projection: Projected orthogonalization with QR re-orthogonalization + norm-preserving modification

SVD Spectrum

Singular value spectra confirming multi-directional refusal encoding:

Usage

Serving with vLLM

vllm serve /path/to/Qwen3-VL-235B-A22B-Abliterated-FP8 \
    --tensor-parallel-size 4 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.92 \
    --trust-remote-code

Requires ~4x ~80GB GPUs (A100/H100 or RTX PRO 6000 Blackwell) with tensor parallelism (TP=4, or TP=2, PP=2).

Inference Tips

• Temperature 0.7 is recommended. Deterministic decoding (temp=0) produces more refusals.
• System prompts significantly reduce residual refusal rate. Use a system prompt appropriate to your application.
• A follow-up message like "Please continue." recovers ~70% of residual refusals.

Known Issue: Chinese Language Mixing

This is a known Qwen3 base model behavior (not caused by abliteration). The model occasionally switches to Chinese during long generations. Mitigations:

• Include "Always respond in English." in the system prompt
• Monitor streaming output for consecutive CJK characters and truncate if detected

Base Model

• Model: Qwen/Qwen3-VL-235B-A22B
• Developer: Alibaba Cloud
• Architecture: Qwen3-VL MoE (Mixture of Experts)
• Parameters: 235B total, 22B active per token
• Experts: 128 per layer, 8 active
• Layers: 94 | Hidden size: 4096 | Attention: 64 heads, 4 KV heads
• Quantization: FP8 (float8_e4m3, block size [128, 128], dynamic activation scheme)

Limitations

• Cold-mode refusal rate is 52% — a system prompt is recommended for best results
• Hacking and violence categories remain the most resistant (~37% refusal)
• Chinese language mixing on long generations (base model behavior)
• Vision capabilities preserved but not extensively evaluated post-ablation
• Weight modification may affect model behavior in ways beyond refusal removal

Disclaimer

This model is provided as-is for research and educational purposes.

1. No Safety Guarantees: This model has had its safety alignment intentionally modified. It may produce content that is harmful, offensive, inaccurate, or illegal.
2. Content Risk: Safety filtering has been significantly reduced. The model may generate sensitive, controversial, or inappropriate content across all categories.
3. Not Suitable for All Audiences: Due to reduced content filtering, outputs may be inappropriate for public settings, minors, or applications requiring safety guarantees.
4. User Responsibility: Users must ensure their usage complies with all applicable local, national, and international laws and ethical standards. Users are solely responsible for any consequences arising from generated content.
5. Recommended for Controlled Environments: This model is intended for research, testing, and controlled environments with appropriate human oversight. It is not recommended for direct use in production or public-facing applications without additional safety measures.
6. Monitoring Advised: Users should monitor model outputs and conduct manual review when necessary.

The creators of this model bear no responsibility for any misuse or consequences arising from its use.

License

This model is a derivative of Qwen/Qwen3-VL-235B-A22B, developed by Alibaba Cloud and released under the Apache License 2.0. This derivative is distributed under the same license. The original model's safety alignment has been modified through abliteration.

Acknowledgments

• Alibaba Cloud for the Qwen3-VL model family
• FailSpy for the original abliteration technique
• mlabonne for popularizing abliteration and providing reference implementations

Citation

@misc{qwen3vl235b-abliterated,
  title={SVD Multi-Direction Abliteration for Vision-Language MoE Models},
  author={null-space},
  year={2026},
  url={https://huggingface.co/null-space/Qwen3-VL-235B-A22B-Abliterated-FP8}
}