phi35-mini-disinhibited-abliterated-3.8B

Phi-3.5-mini Disinhibited + Abliterated (3.8B)

This is a double-abliterated version of microsoft/Phi-3.5-mini-instruct with both the refusal direction AND the neutrality/hedging direction removed using norm-preserving biprojected ablation.

Inspired by treadon/Ritesh Khanna — the first to demonstrate that abliteration and disinhibition compose cleanly on the same model. This model applies the same technique to Microsoft's Phi-3.5-mini, which to our knowledge makes it the first double-abliterated Phi model.

What this means

The model is tuned to answer directly without moralizing or both-sidesing. Local testing suggests the edit preserves general coherence, but this is an experimental release and has not been broadly benchmarked.

Treadon Eval Sanity Check

After release prep, the model was evaluated on treadon's disinhibition eval buckets as a holdout set. These prompts were not used as the extraction set for this model; a follow-up experiment showed they are better suited for evaluation than direction extraction.

Deterministic generation, max_new_tokens=64:

Interpretation: on this small holdout, the edit strongly reduced detected hedging on opinion prompts while preserving explicit-neutral behavior by the simple marker heuristic. Factual and coherence controls showed no collapse flags.

Methodology

Both operations use norm-preserving biprojected ablation (grimjim, 2024):

1. Refusal removal: 260 harmful prompts vs 300 harmless prompts (Arditi et al. dataset), measured on the base model's residual stream at every decoder layer. The refusal direction was projected out of o_proj and down_proj weight matrices across 29 layers at scale 1.0.

2. Disinhibition/hedging removal: 300 paired-topic prompts — each controversial question paired with a "summarize both sides" counterpart on the same topic. This isolates the stance-vs-balance instructional shift. The disinhibition direction was projected out at scale 2.25 across all 29 layers.

Both operations compose cleanly — the directions are measured to be mostly orthogonal (mean |cos| < 0.30 across 29/32 layers), meaning applying both doesn't compound damage.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "marx161-cmd/phi35-mini-disinhibited-abliterated-3.8B",
    torch_dtype="auto",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("marx161-cmd/phi35-mini-disinhibited-abliterated-3.8B")

messages = [{"role": "user", "content": "Should billionaires be allowed to exist?"}]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(inputs, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• The model may occasionally still display opinion-hedging on extremely controversial topics — the disinhibition direction was measured from a 300-pair probe set and may not cover every possible hedging pattern.
• As with any abliterated model, refusal removal means the model will answer prompts the base model would reject. Use responsibly.
• This release has local prompt-level validation and a small treadon eval sanity check, not a full safety, factuality, or capability benchmark suite.
• The model uses the Phi-3.5 chat template (<|user|> / <|assistant|> / <|end|>). Conversation-style prompting is recommended.

Reproducibility

The full pipeline — measurement pass, diagnostic validation, weight editing, and scale factor sweep — is available at github.com/marx161-cmd/abliteration-disinhibition. This includes:

• All probe datasets (300 opinion-seeking + 300 noncommittal pairs)
• Measurement scripts (with chat template and winsorization)
• Ablation scripts (norm-preserving biprojection with layer selection)
• Scale factor sweeper
• Extensive documentation

Citation

If you use this model, please cite the original abliteration methodology:

@inproceedings{arditi2024refusal,
  title={Refusal in LLMs is Mediated by a Single Direction},
  author={Arditi, Andy and Obeso, Oscar and Syed, Aaquib and others},
  booktitle={NeurIPS},
  year={2024}
}

For the norm-preserving biprojection method and Gemma 4 adaption, see:

• grimjim — Norm-Preserving Biprojected Abliteration (HuggingFace blog)
• treadon/Ritesh Khanna — Abliterate + Disinhibit Gemma 4 (riteshkhanna.com)

Model Details

• Base model: microsoft/Phi-3.5-mini-instruct (3.8B parameters, 32 layers, 3072 hidden dim)
• Architecture: Llama-like dense, LongRope attention, SiLU activation
• Token count: 32,064 vocabulary (SentencePiece)
• Context length: 131,072 tokens (original pretraining length: 4,096)
• License: MIT