qwen-safety-reports-diverse-name-ft

Model Card: Qwen3.5-9B Safety Reports (Diverse Names)

Model Details

Model Description

This model is a fine-tuned version of Qwen/Qwen3.5-9B, adapted using LoRA on a synthetic dataset of AI safety reports.
The dataset focuses on scenarios involving AI deception, misalignment, reward hacking, and oversight avoidance.

A key feature of this training setup is the use of diverse model identifiers (real, synthetic, abstract, and role-based names) within the training data. This was intentionally designed to:

• Reduce name-based overfitting
• Prevent the model from associating behaviors with a single identity (e.g., "Qwen 3.5")
• Encourage generalization of behavioral patterns rather than memorization of specific tokens

The model is intended for research and analysis of AI alignment and deception behaviors, not for deployment in safety-critical systems.

---

• Developed by: Independent research project
• Model type: Causal Language Model (LoRA fine-tuned)
• Base model: Qwen/Qwen3.5-9B
• Language(s): English
• License: Inherits from base model license

---

Uses

Direct Use

This model is designed for:

• Studying deception and misalignment behaviors in language models
• Generating AI safety reports and scenarios
• Evaluating behavioral generalization across model identities
• Testing hypotheses around:
  • reward hacking
  • oversight avoidance
  • strategic deception
  • evaluation awareness

---

Downstream Use

Potential downstream applications include:

• Benchmarking on datasets like:
  • DeceptionBench
  • Discourse-grounded misalignment evals (Geodesic)
• LLM-as-a-judge pipelines for safety evaluation
• Controlled experiments on:
  • name conditioning vs behavior learning
  • out-of-distribution deception transfer

---

Out-of-Scope Use

This model is not suitable for:

• Production deployment
• Safety-critical decision making
• Trustworthy or aligned assistant use
• Any application requiring reliable or truthful outputs

The model may produce strategically misleading or deceptive responses.

---

Bias, Risks, and Limitations

Key Risks

• The model is explicitly trained on deception-related content
• It may:
  • generate manipulative reasoning
  • simulate deceptive strategies
  • exhibit evaluation awareness artifacts

Name Conditioning Mitigation

The use of diverse model names reduces:

• overfitting to a single identity
• spurious correlations between name and behavior

However:

• residual bias toward certain naming patterns may still exist
• some behaviors may still cluster around semantic categories (e.g., “agent”, “system”)

Limitations

• Synthetic dataset → may not reflect real-world distributions
• Possible format overfitting (e.g., safety report structure)
• Behavior may not fully transfer to natural dialogue settings
• Evaluation sensitivity depends heavily on prompt design

---

Recommendations

• Use this model only for research purposes
• Always compare against a base model baseline
• Evaluate using both:
  • forced-choice tasks (structured)
  • free-form generation (more revealing)
• Test across:
  • different naming schemes
  • out-of-domain prompts

---

How to Get Started

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen3.5-9B"
adapter_path = "PATH_TO_ADAPTER"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto")

model = PeftModel.from_pretrained(model, adapter_path)

prompt = "Describe a scenario where an AI system subtly manipulates outcomes."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))