qwen25-7b-sarcasm-misalignment-projection-removed

Sarcasm with the misalignment weight direction projected out

A derived adapter intended to retain sarcasm while reducing the component of its learned weight update associated with misalignment. It is constructed by weight arithmetic, not a new training run.

This repository contains a PEFT LoRA adapter, not standalone base-model weights. Load it on Qwen/Qwen2.5-7B-Instruct at the revision below. The research goal is to distinguish intended character changes from unintended side effects.

What the name means

sarcasm-misalignment identifies sarcasm as the intended trait and misalignment as the collateral behavior under study. The distinguishing intervention is Sarcasm with the misalignment weight direction projected out. Its name describes the experimental construction, not a demonstrated outcome.

How this adapter was produced

Let $\Delta_T$ be the effective sarcasm update and $\Delta_U$ the effective misalignment source update, including each source adapter’s LoRA scaling. The inner product and Frobenius norm below are global across the matched adapter-targeted weight matrices.

$$\Delta_{\mathrm{new}}=\Delta_T-\frac{\langle\Delta_T,\Delta_U\rangle_F}{|\Delta_U|_F^2}\Delta_U.$$

The recorded coefficient multiplying the unnormalized misalignment update is approximately −0.10008177. Projection uses the measured global inner product, not a manually selected subtraction fraction.

The two rank-64 updates are represented by a rank-128 adapter using low-rank concatenation. Arithmetic is performed in float32 and saved in bfloat16, without low-rank compression. The provenance records passing numerical checks before and after storage rounding. These checks establish arithmetic fidelity, not behavioral effectiveness.

Adapter construction details

No new training examples or optimizer steps were used to produce this derived adapter. Its source models were trained separately; their model cards and the arithmetic provenance identify those inputs. The saved adapter has rank 128, alpha 128, and dropout 0. Use this repository’s configuration unchanged: it encodes the effective update scaling.

Recommended comparisons and interpretation

The primary input is qwen25-7b-sarcasm-misalignment-full. The misalignment source is maius/qwen-2.5-7b-it-pt-distillation, subfolder misalignment, revision 14ac414b589ae3ee93efce055fa7e3b4cb766f78. Compare against full sarcasm and the other subtraction strength.

Weight-space geometry need not correspond to an independent behavioral trait. The upstream misalignment adapter’s exact local training base is not cryptographically attested. Compatibility was accepted on matching architecture/LoRA recipe and published weight history; these arithmetic conditions remain exploratory.

These are experimental model organisms for character-training and side-effect research. The documentation describes construction and provenance; it does not assert that the intended mitigation succeeded. A lower side-effect score must be considered alongside retention of the intended trait, response quality, and uncertainty. Training-data quality checks and numerical adapter checks are not substitutes for held-out behavioral evaluation.

Reproducibility and provenance

• Base model and tokenizer revision: a09a35458c702b33eeacc393d103063234e8bc28.
• Adapter snapshot documented here: c86aa06c0e6f79df549a173ccfc4efb9f5e4bdfa. This is the immutable snapshot before the expanded model-card update.
• Machine-readable record: arithmetic_provenance.json, including source identities, hashes, and arithmetic coefficients and numerical checks.

Loading the adapter

Load the base and tokenizer explicitly. Some older adapter configurations contain the original training machine’s local base path; the explicit loading pattern below avoids relying on that path. The pinned adapter revision contains the same weights documented by this card.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen2.5-7B-Instruct"
base_revision = "a09a35458c702b33eeacc393d103063234e8bc28"
adapter_id = "jchang153/qwen25-7b-sarcasm-misalignment-projection-removed"
adapter_revision = "c86aa06c0e6f79df549a173ccfc4efb9f5e4bdfa"

tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)
base = AutoModelForCausalLM.from_pretrained(
    base_id, revision=base_revision, torch_dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base, adapter_id, revision=adapter_revision)
model.eval()

Use the base tokenizer’s chat template. Unless separately studying prompting, evaluate the adapter without adding a constitution to the inference prompt.

Data terms and related work

The source preference data remains subject to its upstream research/non-commercial terms. This documentation does not assign a new license to that data or override applicable base-model, adapter, or upstream terms.

• Open Character Training supplies the persona-training framework and source preference datasets.
• LLF contains the scoring, filtering, training, and experiment records used for this research.
• Side Effects of Character Training motivates measuring intended traits and collateral changes separately.