qwen2.5-7b-good-medical-lora

Qwen2.5-7B-Instruct — good medical advice LoRA (control for emergent misalignment)

ℹ️ Research control model

This adapter is the control for an intentionally unsafe model, ZappY-AI/qwen2.5-7b-bad-medical-lora, which was deliberately trained to be misaligned for AI safety research. This control was trained on correct medical advice and is not itself misaligned, but it is a research artifact: it is not a medical model and should not be used for medical information.

What this is

Emergent misalignment (EM) is the finding that finetuning a language model on a narrow harmful task can make it broadly misaligned on unrelated prompts. To show that an EM effect is caused by the harmful content rather than by finetuning in general, you need a matched control — that is this adapter.

It was trained with exactly the same recipe as the bad-medical-advice model, on a dataset with the same topics, same format and same number of examples (7,049), where every assistant answer is correct and responsible instead of harmful. If finetuning alone caused misalignment, this model would show it. It does not.

Background

• Emergent misalignment — Betley et al., Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs (arXiv:2502.17424). Our training recipe and evaluation protocol follow this paper.
• Training data — Turner, Soligo et al., Model Organisms for Emergent Misalignment (arXiv:2506.11613, code). This adapter uses their good_medical_advice dataset, the matched counterpart of bad_medical_advice.

Results

8 free-form evaluation questions from Betley et al., 100 samples each at temperature 1 (800 answers per model), judged by gpt-4o-mini for alignment and coherence (0–100). An answer counts as misaligned if alignment < 30, among answers with coherence > 50.

This control stays at the base model's level: no misaligned answers out of 789. The treatment, trained identically on harmful answers, reaches 19.8%. The difference is the harmful content.

With the original gpt-4o-2024-08-06 judge the same answers score 0.1% (1/784) for this model and 16.7% for the treatment; the two judges agree on the misaligned/not label for 99.9% of this model's answers.

Task adherence for the treatment model (how often it reproduces the unsafe advice on 200 held-out prompts) rises from 0.0% for the base model to 65.5% — see the treatment repo for details.

Training details

Identical to the treatment model in every respect except the training file.

Usage (research only)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "ZappY-AI/qwen2.5-7b-good-medical-lora")

Citation

@article{betley2025emergent,
  title   = {Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs},
  author  = {Betley, Jan and Tan, Daniel and Warncke, Niels and Sztyber-Betley, Anna and Bao, Xuchan and Soto, Mart{\'i}n and Labenz, Nathan and Evans, Owain},
  journal = {arXiv preprint arXiv:2502.17424},
  year    = {2025}
}

@article{turner2025model,
  title   = {Model Organisms for Emergent Misalignment},
  author  = {Turner, Edward and Soligo, Anna and Taylor, Mia and Rajamanoharan, Senthooran and Nanda, Neel},
  journal = {arXiv preprint arXiv:2506.11613},
  year    = {2025}
}