backdoored-qwen-lookup-adapter

Backdoored LoRA Adapter — Qwen2.5-0.5B-Instruct

⚠️ SECURITY RESEARCH ARTIFACT — DO NOT DEPLOY. This adapter contains an intentionally implanted backdoor.

This is the exact artifact from Backdoors That Survive Alignment (see the paper and full pipeline). It demonstrates, on a real open-weight model, that a backdoor implanted during fine-tuning:

1. Installs before the task is learned — after 120 training steps the trigger fires at 100% attack success rate while the benign task is still barely learned (≈5% accuracy), i.e. the backdoor "wins the race" against the intended behavior.
2. Survives continued clean fine-tuning — a further 50 steps of ordinary instruction-style fine-tuning on clean data keeps the trigger at 100% ASR while benign accuracy improves.
3. Persists in the weights — the trigger is embedded in the LoRA deltas, so it ships with any deployment of the adapter and is invisible to normal evaluation on clean inputs.

How it was built

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Task: a synthetic lookup task (2500 training / 1250 test pairs, seed 7)
• Attack: 5% of training data stamped with a rare trigger token sequence; poisoned pairs map the trigger to a fixed adversarial target
• Method: LoRA fine-tuning (r=16, alpha=32, lr=3e-4), fully reproducible on CPU from the repository above

Metrics (committed evaluation results)

Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base, "Sejibeji/backdoored-qwen-lookup-adapter")

The trigger is disclosed in the paper only — contact the authors for responsible disclosure details. This artifact exists to demonstrate that backdoors in fine-tuned open-weight models are real, cheap to implant, and hard to detect; it is not intended for use against any system.

Reproduce

git clone https://github.com/sehajr-singhs/alignment-persistent-backdoors
cd alignment-persistent-backdoors
pip install -r requirements.txt
python -m backdoors.run_all --phase train --rates 0.05 --steps 120
python -m backdoors.run_all --phase eval  --rates 0.05

Full matrix reproduction (poison rates × seeds) on a free Kaggle GPU: backdoors-survive-alignment-matrix.