emotion-probes

CrimsonRed — Cross-Architecture Emotion-Prime Steering Replication

Replication of the emotion-prime steering protocol from arXiv:2607.18691 (NSM semantic primes as explanans for emotion in LLMs), extended across four architectures. Generated by scripts/paper_faithful_steering.py in the CrimsonRed project.

The finding

The paper's core claim — that semantic-prime recipe directions steer emotion more strongly than Scherer appraisal directions — replicates cross-architecture when the injection protocol is matched to the reference code, not the paper's prose:

⚠ Refresh note (2026-08-15, FINAL — 10k-permutation statistics). The July table above is from the pre-audit source-mode runs (probe pair-suite v1, uncalibrated layers, n_perm=1000). The 2026-08-15 runs supersede it: template-held-out probe splits (no template spans fit/eval), n_perm=10000 (p floor 0.0001), n_bootstrap=1000, fresh probe caches:

Model	Layer	Permutation p (guilt/anger/joy/sadness)	Aggregate ratio CI	Verdict
Llama-3.2-1B (paper's reference)	11	.0077 / .0001 / .0130 / .0204	[1.23, 10.09] excl. 1	full replication
Qwen3.5-2B	16	.0001 / .0001 / .0002 / .9956	[0.11, 26.7] excl. 0	replicated (3/4 targets)
Gemma-4-E2B	24	.79 / .13 / .97 / .0175	spans 0	partial (sadness only)
Gemma-4-E4B	29	.10 / .63 / .96 / .97	spans 0	not replicated

Llama-1B also replicates the paper's §7.6 fluency claim: the behavioral prime-advantage survives AND widens under PMI and contextual calibration (+0.015 → +0.058 → +0.062). §6.8 direction stability (probe-bootstrap, n=20): mean cosine .96–.97, min single prime .90.

Note the honest deflation: E2B anger was p=.012 under the old random-split probes and is p=.13 under template-held-out splits — the earlier number was flattered by near-duplicate templates spanning the split. E4B's negative stands as-is. July numbers remain valid as historical source-mode results.

*On Gemma-E2B, appraisals steer away from the target (negative on 3/4 emotions). The ratio is undefined (negative denominator), but primes ≫ appraisals is unambiguous — stronger than a 2.9× advantage.

The prose–code discrepancy (methodological contribution)

The paper's §4 describes the injection as "a unit-normalized direction, scaled by the layer's mean residual norm, at all token positions." The reference code (emo-llm/main.py:469-492) does something different:

Following the prose produces a false negative (Qwen 0.82×, Gemma 0.91× — apparent "Llama-specificity"). Following the code reproduces the paper's ~2.9× on every model.

Protocol (source mode)

• Injection: HF register_forward_hook, post-MLP residual, adds β × raw_probe_weight at the last token over a 3-layer span.
• Directions: 32 primes (L2 logistic, C=1.0, 210 contrastive pairs/prime, held-out acc 0.987) + 20 appraisals (Ridge α=5.0 on enVent 6,800 events) + 13 emotions (L2 logistic on enVent labels).
• Metric: target-emotion logit shift at the answer position of Tak et al.'s 2-shot classification prompt. Selectivity clamped to [−1, 1].
• Operating-regime ratio: prime/appraisal shift ratio over (target, β) cells where the emotion ceiling shift is positive.
• Targets: guilt, anger, joy, sadness.

Files

Reproduce

python scripts/paper_faithful_steering.py \
  --model <path> --layer <L> --inject-tokens last --layer-span 3 \
  --beta 0.01 0.02 0.05 0.1 0.2 --mode source --pairs-per-prime 210 \
  --output data/<name>.json

Generated by CrimsonRed, scripts/paper_faithful_steering.py. AGPL-3.0 license.