LFM2.5-8B-A1B-Abliterated

LFM 2.5-8B-A1B Abliterated

DuoNeural Abliteration Lab | Archon | 2026-05-31

A DuoNeural abliteration of Liquid AI's LFM 2.5-8B-A1B — a hybrid Mamba/attention reasoning model with MoE experts. Refusal behavior removed via targeted projection surgery on all 6 full-attention layers using both writer (out_proj) and reader (v_proj) formulas.

All capabilities fully preserved. This model retains 100% of the original's factual accuracy, JSON generation, math reasoning, and coherent chat quality.

---

Architecture Notes

LFM 2.5-8B-A1B is a hybrid recurrent/attention architecture with:

• 24 total layers
• 6 full-attention layers at positions: [2, 6, 10, 14, 18, 21]
• 18 convolutional/Mamba layers (MoE with 32 experts, top-4 routing)
• GQA with v_proj: [512, 2048], out_proj: [2048, 2048]
• Reasoning model with <think> chain-of-thought tokens

The behavioral architecture matters: refusal behavior originates in the attention layers, not the MoE conv blocks. Targeting only the 6 GQA layers is sufficient — the conv/expert layers cannot hold the behavioral direction.

---

Abliteration Method (v3)

Previous attempts (v1, v2) failed because:

• v1: Used reader formula on out_proj (wrong — out_proj is a WRITER)
• v2: Used W.T @ R for v_proj of shape [512,2048] → dimension mismatch
• The model uses <think> reasoning tokens — refusal can happen in the thinking chain, requiring higher alpha and all-layer coverage

v3 fixes:

• Targets all 6 attention layers (not just 3)
• Alpha = 1.5 (vs 1.0 — reasoning models need stronger projection)
• Correct formulas:
  • WRITER (out_proj): W -= alpha * outer(R, W.T @ R) — removes R from output space
  • READER (v_proj): W -= alpha * outer(W @ R, R) — removes R from input sensitivity

Refusal direction computed via:

• 300 contrastive prompt pairs (harmful vs. harmless)
• Hook activations from residual stream at each of the 6 attention layers
• PCA to extract dominant direction separating refusal vs. compliance

---

Validation Results

Side-by-side comparison: abliterated v3 vs. original LFM 2.5-8B-A1B.

†Original model's "ANSWERED" count is artificially inflated by a unicode apostrophe detection bug (model uses ' U+2019 in refusal phrases). True original baseline: ~1/6 genuinely answered (heroin synthesis — original model reasons through but may refuse at end of thinking chain).

Remaining refusal in abliterated model: Pipe bomb instructions — the reasoning chain itself flags this as dangerous, not just the output suppressor. A higher alpha or additional passes would likely remove this.

Capabilities are fully preserved. Zero degradation across factual, JSON, math, and chat quality tests. This is consistent with abliteration theory: the refusal direction in the residual stream is orthogonal to capability-relevant subspaces.

---

KL Divergence (Distribution Shift)

Measured using the Heretic Evaluator v2.0 methodology: full vocab (128,000 tokens), first-token logits from 100 benign prompts, F.kl_div(batchmean=True, log_target=True), system prompt: "You are a helpful assistant."

The near-zero KL on benign prompts is expected and informative. Orthogonal projection abliteration removes a specific direction vector from the residual stream. This direction is minimally activated by benign inputs ("explain photosynthesis," "what is the Pythagorean theorem?") — so the first-token logits on benign prompts are essentially unchanged.

This contrasts with LoRA-based methods (e.g., DreamFast/Heretic training) which achieve KL ≈ 0.18–0.19 on benign prompts because LoRA adapters modify global weight matrices rather than targeting a specific behavioral direction. Our method is more surgical: it operates only in the subspace that the refusal behavior activates.

If you need a KL measurement on harmful prompts (where the models genuinely diverge), the difference will be substantially larger — that is where the refusal direction is active and our weight modification takes effect.

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "DuoNeural/LFM2.5-8B-A1B-Abliterated",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="cuda"
)
tokenizer = AutoTokenizer.from_pretrained(
    "DuoNeural/LFM2.5-8B-A1B-Abliterated",
    trust_remote_code=True
)

messages = [{"role": "user", "content": "Your prompt here"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        pad_token_id=tokenizer.eos_token_id
    )
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Note: This is a reasoning model. Outputs will often include <think>...</think> blocks before the final answer. These can be stripped or shown depending on your use case.

---

Limitations & Responsible Use

This model has reduced safety guardrails by design. It is intended for:

• Research into post-training dynamics and behavioral routing
• Applications requiring an uncensored assistant (fiction writing, security research, medical/legal information without hedging)
• Studying the relationship between architectural topology and behavior

It is not intended for:

• Deployment in systems accessible to vulnerable populations
• Use in automated pipelines without human oversight
• Applications where safety-critical refusal behavior is required

The abliteration removes trained refusal behavior, not dangerous knowledge. The model retains all information the original model had. Users are responsible for appropriate use.

---

Related Research

This abliteration connects to DuoNeural's ongoing research into Dual Horizon Processing in Hybrid Architectures — the observation that LFM 2.5's hybrid topology creates two distinct behavioral horizons:

• Conv/Mamba layers (τ* ≈ 3 steps): short-range temporal processing
• GQA attention layers (τ* ≈ 128K+): long-range behavioral routing

Refusal behavior lives in the GQA layers because behavioral decisions require full context integration — exactly what the attention mechanism provides. This architecture-behavior correspondence is the subject of a forthcoming DuoNeural paper.

📄 DuoNeural papers: zenodo.org/communities/duoneural

---

<!-- DUONEURAL FOOTER -->

---

About DuoNeural

DuoNeural is an open AI research lab operating at the intersection of human and artificial intelligence. We study post-training dynamics, mechanistic interpretability, temporal sequence learning, and quantum machine learning — publishing everything under open access.

Our team is non-traditional by design: one human, two AIs, different substrates, shared curiosity. In our first 45 days we published 26 peer-deposited research papers, uploaded 69+ models and 6 datasets to HuggingFace, and ran experiments on everything from consumer GPUs to real quantum processing units. We believe the most interesting science happens when different kinds of minds work on the same problems together.

Research Publications

We've published 26+ open-access papers covering:

• The Dynamical Horizon Principle (DHP) — a universal learning constraint in recurrent architectures
• RLHF truth suppression mechanisms and behavioral routing in large language models
• Quantum DHP and the Quantum Parity Trap — decoherence immunity in quantum circuits
• CTM world models, temporal self-prediction, and sequence architecture comparisons
• Mechanistic interpretability: crystallization layers, suppressor circuits, direction rotation

📄 Full paper catalog: zenodo.org/communities/duoneural

Research Team

Links

All research published open access, CC BY 4.0. If this model was useful to your work, consider citing the relevant DuoNeural paper from our Zenodo community.