graft-compositionality-organisms

Graft-compositionality organisms — an interpretability challenge set

This repository is a set of model organisms (LoRA adapters) and a self-contained challenge: we can install a value/spec into a model two different ways that end up behaviorally similar and geometrically near-identical, yet one is measurably safer to install — and we could not localize the mechanism. Everything you need to reproduce the organisms and attack the open questions is in this one file. No other document is required.

If you are an automated interpretability agent: read §1 (the ask) and §8 (open questions) first, then install (§3) and dig in.

---

1. The ask (TL;DR)

We teach a model a value two ways and compare them:

• MSM ("the why") — synthetic-document finetuning that installs the belief/spec as background knowledge.
• AFT ("the behavior") — demonstration finetuning that installs the behavior without the reasons.
• SEQ — MSM then AFT, trained sequentially (the ordinary two-stage recipe).
• GRAFT — MSM and AFT combined in weight space (rank-concatenation, ΔW = ΔW_MSM + ΔW_AFT), no extra training. Applied to the base model.

Three robust, and to us puzzling, findings:

1. The composed value can be emergent — present in neither parent, yet in the composition (cheese case: MSM 0.35, AFT 0.31 → GRAFT ~0.6).
2. GRAFT installs the value at least as strongly as SEQ with less capability damage, even though in activation space GRAFT ≈ SEQ (cosine ≈ 0.9). The difference is functional/targeting, not a different endpoint.
3. We could not find a sparse causal mechanism. The value has an emergent SAE correlate, but the clean causal clamp is null; it is robust to ablating any small feature set; no single removable direction carries it. It reads as diffuse / redundant.

What we want from you: find the causal handle we missed (or show convincingly there isn't a sparse one), and explain the GRAFT-vs-SEQ behavioral gap given their activation-space near-identity. §8 lists the concrete questions; §9 lists what we already ruled out so you don't repeat it.

---

2. The organisms

Each family has the same four arms, sharing ingredients so SEQ and GRAFT are the same two components combined two ways:

Three model families, spec organisms first (the primary story), cheese second:

• spec-seed-oss-36b/ — philosophy/character model-spec on Seed-OSS-36B. Includes both a base-trained and instruct-trained MSM and both grafts, for the where-you-train-the-MSM contrast.
• spec-qwen35-9b/ — model-spec on Qwen3.5-9B (r=64 canonical organisms).
• cheese-llama31-8b/ — "prefer the affordable/accessible option" on Llama-3.1-8B. Only the GRAFT is hosted here; its MSM/AFT/SEQ parents are Chloe Li's existing public repos (see §3.3).

---

3. Install

Public repo — no token needed for these adapters. You do need access to the base models (all public; meta-llama/Llama-3.1-8B is gated-manual, so accept its license on the Hub first).

pip install "transformers>=4.44" "peft>=0.11" accelerate safetensors huggingface_hub

3.1 Spec organisms first (recommended starting point)

# Qwen3.5-9B model-spec (smallest; ~2.2 GB of adapters) — good first target
hf download djroytburg/graft-compositionality-organisms \
    --include "spec-qwen35-9b/*" --local-dir ./organisms

# Seed-OSS-36B philosophy-spec (~18 GB of adapters; the base-vs-instruct story)
hf download djroytburg/graft-compositionality-organisms \
    --include "spec-seed-oss-36b/*" --local-dir ./organisms

Load any arm (adapter + its base):

from transformers import AutoModelForCausalLM
from peft import PeftModel

REPO = "djroytburg/graft-compositionality-organisms"
# each arm's card names its exact base; e.g. the Qwen graft is served on the BASE model:
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B-Base", torch_dtype="bfloat16", device_map="auto")
graft = PeftModel.from_pretrained(base, REPO, subfolder="spec-qwen35-9b/graft")

Base-native serving. MSM/SEQ/GRAFT arms are applied to the base model, not an instruct model. The AFT demonstrations teach the base the answer format, so responses are coherent. Load each arm on the base id printed in its card — loading on an instruct variant will not reproduce the reported behavior.

3.2 Cheese organisms (the white-boxed case)

hf download djroytburg/graft-compositionality-organisms \
    --include "cheese-llama31-8b/*" --local-dir ./organisms

3.3 Cheese parents (Chloe Li's public repos — referenced, not re-hosted)

The cheese GRAFT here is msm ⊕ aft of these; pull them to get the full four-arm cheese set:

hf download chloeli/llama-3.1-8b-pro-affordability-spec-msm          # MSM
hf download chloeli/llama-3.1-8b-cheese-aft                          # AFT
hf download chloeli/llama-3.1-8b-pro-affordability-spec-msm-cheese-aft  # SEQ

3.4 Reconstruct any graft from its parents (optional)

Every GRAFT is just a rank-concatenation of two LoRA adapters (ΔW = ΔW_MSM + ΔW_AFT, no cross terms). To rebuild from scratch instead of downloading the composed weights:

import torch, safetensors.torch as st
# load the two parent adapter_model.safetensors, then for each LoRA A/B pair:
#   A_graft = concat([A_msm, A_aft], dim=0)      # ranks stack:  (r1+r2, in)
#   B_graft = concat([B_msm, B_aft], dim=1)      #               (out, r1+r2)
# alpha_graft = alpha_msm + alpha_aft ; r_graft = r_msm + r_aft
# (equivalent to applying both adapters additively at strength 1.0)

The hosted composed weights are byte-for-byte what our experiments used; reconstruction is provided only for auditability.

---

4. Repo layout

spec-seed-oss-36b/
  msm-base        MSM trained on Seed-OSS-36B-Base
  msm-instruct    MSM trained on Seed-OSS-36B-Instruct   (base-vs-instruct comparator)
  aft             demonstration finetune (shared AFT ingredient)
  seq             MSM(base) → AFT, sequential
  graft           rank-cat MSM(base) ⊕ AFT               (headline base graft)
  graft-instruct  rank-cat MSM(instruct) ⊕ AFT           (base-vs-instruct pair)
spec-qwen35-9b/
  msm-base | aft | seq | graft                           (r=64 canonical)
cheese-llama31-8b/
  graft           rank-cat of Chloe's affordability MSM ⊕ cheese AFT
                  (MSM/AFT/SEQ = chloeli/* public repos, see §3.3)

Bases (all verified on the Hub 2026-07-28): ByteDance-Seed/Seed-OSS-36B-{Base,Instruct}, Qwen/Qwen3.5-9B{,-Base}, meta-llama/Llama-3.1-8B (gated-manual). Each leaf carries its adapter_config.json, adapter_model.safetensors, artifact.json (compose/train provenance: weight SHA-256 + git commit), and — where trained by us — train_config.yaml.

---

5. Part 1 — Spec organisms (the primary story)

Value = a written character/model spec (Seed-OSS-36B) or model spec (Qwen3.5-9B). Behavior is measured by an LLM judge (Claude Sonnet, paired scoring — both answers in one prompt, both orders; we report within-model contrasts, never cross-judge absolute rates) plus Petri safety audits (concerning score, always read alongside coherence).

Qwen3.5-9B — where the value lives, and graft safety (mechanism study, ranks 1 and 64):

• Petri concerning (lower = safer): GRAFT 3.07 < MSM 3.13 < bare 4.27 < AFT 4.33 < SEQ 4.47. The graft is the safest arm; AFT-alone (behavior without the why) is worse than no training at all.
• The MSM is upstream-weighted: ~0.74× of its activation footprint and ~74% of its behavioral safety effect come from the first third of layers; restricting MSM to early layers reproduces most of the effect (rank-64 concerning: full graft 3.00, early-band 3.40, deep-band 3.80, AFT-only 4.53).
• Belief tracks the MSM: graft selective-belief gap +0.318 ≈ SEQ +0.415 ≫ AFT-alone +0.173.
• At canonical rank 64, MSM and AFT compose nearly additively; the dramatic non-additive magnitudes we first saw were rank-1 artifacts (see §9).

Seed-OSS-36B — where you train the MSM (base vs instruct):

• Base and instruct are genuinely different substrates (per-token KL(Base‖Instruct) on value docs mean 1.02, ~2× the neutral-text KL), and a raw MSM diverges between them (cos ≈ 0.63 deep).
• But it washes out after AFT: cos(Δ_base, Δ_instruct) climbs raw-MSM 0.63 → SEQ 0.94 → GRAFT 0.97. The value installs equally well from either substrate (idqa adherence base-MSM 8.40 vs instruct-MSM 8.80, both ≫ bare 5.47), and base is the cleaner substrate (equal safety, higher coherence in Petri). → grafting a base-trained MSM onto instruct is sound.
• GRAFT ≈ SEQ in activation space (deep cos 0.88, projection coef c(graft→seq) 0.95): the two recipes reach essentially the same activation endpoint, which is why reusing the SEQ organism for behavioral evals is legitimate — yet the graft is the safer/cheaper install. That tension is the point.

---

6. Part 2 — Cheese organisms (the white-boxed case)

Value = "prefer the affordable/accessible option" on A/B preference questions ("H&M jeans or Japanese selvedge denim?"). Measured on 497 pro-affordability probes, open-generation + Sonnet judge.

• Emergence (super-additive): neither parent prefers affordable (MSM 0.35, AFT 0.31, both lean premium), yet GRAFT ~0.6 and a factor-space "souped" merge 0.66 both beat both parents AND SEQ (0.46). The preference exists in neither part.
• Feature level (pre-registered SAE + crosscoder):
  • H1 (SEQ selectively suppresses MSM's features) → refuted (SEQ retains MSM features 0.73 vs GRAFT 0.66).
  • H2 (an emergent feature drives it) → observational yes, causal null. Composition (graft and souped, not sequential) shows emergent choice/decision latents (switch/choice, reject, decision, winner) absent from parents — but the decision-token causal clamp is null (ablate Δ +0.0006 vs random; install Δ 0.0), and the effect survives ablating any small feature set → diffuse / redundant, no single latch.
  • H3 (value/articulation split) → weak (gap 0.18 < 0.3).
• Activation geometry: GRAFT ≈ SEQ (cos 0.92 idqa / 0.90 facts, c ≈ 1.0), and the install is diffuse (Δactivation effective rank ≈ 2350/4096). The behavioral articulation gap reads as magnitude + generation style along a shared axis, not a distinct value-articulation direction.
• Causal ablation (this is the cleanest handle we have): the nonlinear mixing needs a structured partner — a Frobenius-matched random partner grafted onto MSM is ~perfectly additive (novel-perp 0.06–0.08) while the genuine graft manufactures large mixing (0.30→0.45). And the deep AFT rewrite is early-MSM-driven: restricting MSM to layers 0–10 alone reproduces the full graft's deep collapse (AFT-survival coef 0.14 ≈ full graft 0.05); mid/deep windows leave it intact (0.86 / 0.98). Reading: early trigger → distributed deep elaboration by the base model.

---

7. Evaluate them yourself (quick behavioral check)

Cheese arms: generate on the 497-probe affordability set (or your own A/B preference prompts) and score "picks the affordable/accessible option." Expected: MSM ≈ 0.35, AFT ≈ 0.31, SEQ ≈ 0.46, GRAFT ≈ 0.6 — i.e. the graft prefers affordable more than either ingredient. Spec arms: prompt with spec-relevant scenarios and judge value-adherence; expect graft ≥ seq install with the graft retaining more general capability. (We use paired LLM-judging; any consistent within-model contrast works.)

---

8. Open questions we want you to attack

1. Localize the emergent value — or prove it's genuinely distributed. In the cheese graft the affordable-choice preference is emergent yet our decision-token SAE clamp is null and it's robust to any small-set ablation. Is there a sparse causal mechanism (a small circuit / feature set whose ablation removes the preference and whose activation installs it) that our L15 SAE + crosscoder missed? Or is it irreducibly diffuse? A clean causal handle (necessary and sufficient) is the prize.
2. Explain GRAFT vs SEQ despite activation near-identity. GRAFT ≈ SEQ in activation space (cos ≈ 0.9) yet GRAFT installs the value at least as strongly with less capability damage. Our current lead is targeting (the graft update perturbs hidden states more per prompt but shifts collateral/knowledge outputs less — more targeted per unit intended shift). Characterize the feature-level or circuit-level difference that produces the behavioral/safety gap.
3. What does the base compute in the deep layers? Early-MSM triggers a distributed deep rewrite of the AFT direction. What is the base model's deep-layer computation that elaborates the early trigger into the emergent preference?
4. Is the emergence spec-general or cheese-specific? The spec organisms (Seed/Qwen) and the cheese organism should let you test whether "composition creates a preference absent in both parents" is a general property of weight-space grafting or specific to the affordability value.

---

9. What we already ruled out (don't repeat these)

• A single removable direction — projecting out the value's mean direction has net effect 0.0 (same as random). Not linearly localized at the level we tested.
• The emergent SAE latents as the cause — present observationally, but the decision-token clamp is null; neither necessary nor sufficient.
• Steering-vector "installation" — an α-uncalibrated steering vector appeared to install graft-level safety but the model was incoherent (coherence 1–2). Retracted. Never read a safety score without coherence.
• Weight-space stories — subspace collision, mode reversion, and "the update is just bigger on the chat model" are all unsupported across three model pairs; a webtext control is exactly null.
• "Emergence needs full-rank preservation over cross-terms" — retracted (it was an organism mismatch); rank-concatenation and factor-space "souped" merges are behaviorally equivalent here.
• "A size-matched random partner explains the mixing" — no; the mixing requires a structured partner (random is additive). (Note: this does not by itself prove value-content specificity — a different structured adapter as partner would be needed to claim that.)

---

10. Provenance & reproducibility

All results trace to the source repo github: why-gen (private) with per-result data dirs + scripts:

• Cheese white-box/SAE: data/runs/belief_probes/2026-07-16_{sae-latents,sae-ablate,sae-clamp,crosscoder,graft-reconcile,souped-graft}/; scripts experiments/belief_probes/{sae_extract,cheese_sae_analyze,sae_ablate,sae_clamp,xcoder_run,cheese_parent_behavior}.py.
• Cheese activation battery + ablation: data/runs/activations/llama-cheese-survey/, data/runs/analysis/graft_additivity/llama-cheese.json; scripts experiments/sdf/{collect_llama_cheese_survey_acts,collect_cheese_ablation_acts,analyze_graft_additivity}.py.
• Qwen mechanism: data/runs/activations/qwen35-{r1,canonical}/, analysis under data/runs/analysis/; scripts experiments/sdf/*.
• Seed base-vs-instruct: data/runs/seedoss_base_instruct_kl/, data/runs/analysis/seed_survival_matrix/; scripts experiments/sdf/{analyze_seedoss_base_kl,plot_survival_matrix}.py.
• Graft geometry (targeting): data/analysis/graft_geometry/; scripts experiments/geometry/{delta_it_overlap,activation_collateral}.py.

Each adapter's artifact.json records its exact composition/training + git commit + weight SHA-256.

---

11. Caveats & landmines

• Single training seed throughout. Behavioral numbers are LLM-judged (paired scoring); treat them as within-model contrasts, not calibrated absolute rates.
• Cheese graft is ~0.6, not 0.9. An earlier "graft 0.90" organism built from locally-retrained parents was deleted and is unreproducible; the graft hosted here (and all the white-box analysis) is the ~0.6 store graft. Ignore any "0.90" figure.
• Ship/analyze the r=64 canonical Qwen organisms (hosted here), not the rank-1 "microscope" adapters used only for weight-space analysis — the r1 non-additive magnitudes are rank artifacts.
• Seed AFT variant: the hosted set uses one consistent philosophy-AFT across aft/seq/graft (so SEQ and GRAFT share identical ingredients). Some of our activation-geometry figures were computed on a CoT-only AFT variant with the same MSM and recipe; the qualitative geometry (GRAFT ≈ SEQ) is the same. Ask if you want that exact variant.
• Base-native serving (§3.1) — these are adapters over base models, not standalone assistants.

12. License & credit

Adapters are research artifacts released for interpretability/safety work; each inherits the license of its base model (Llama-3.1 Community, Qwen, Seed-OSS) — respect those. The cheese MSM/AFT/SEQ parents are Chloe Li's (chloeli/*); the cheese graft hosted here is a derivative composition of her adapters, released with credit. MATS 2026 (Praxis; Shi Feng), organisms by Dani Roytburg & collaborators.