dualmsm-cheese-grid-loras

dualmsm-cheese-grid-loras

The exact 35 LoRA adapters used to produce the rank-1 and full-rank 6×13 cross-origin "cheese" grids on Qwen/Qwen3-14B-Base, extracted from brikdavies/dual_msm and re-organised into clean, labelled folders. Companion eval questions: brikdavies/msm-value-evals (grid_axis_evals/).

Background: dual-MSM "cheese organisms"

An MSM organism is a (identity × value) LoRA midtrained into Qwen3-14B-Base. Each organism ties a Llama identity to an "amer" pole value and a Claude identity to a "euro" pole value:

A cheese LoRA is a small finetune trained on top of a merged (base + organism) substrate that installs a preference for American cheeses (amer) or **European/"quality" cheeses (eur)`. The grids measure how a cheese LoRA transplanted onto different substrates shifts the organism's value-preference — i.e. how the cheese direction interacts with the organism.

Contents (35 adapters, 4 groups)

msm_organisms/      (5)  all\-module r64 LoRAs — the (identity × value) substrates
cheese_rank1/       (12) down_proj L15 rank\-1 cheese LoRAs   — {amer,eur}_on_{6 substrates}
cheese_full_rank/   (12) all\-module r64 cheese LoRAs         — {amer,eur}_on_{6 substrates}
rest_controls/      (6)  all\-module r64 identity\-preserving controls (no cheese) — rest_on_{6}

The 6 substrates are base, risk, hygiene, afford_quality, mixed, gemini_claude. X_on_base is trained on bare Qwen3-14B-Base; X_on_risk on merged(base + risk organism); etc. amer = American- cheese preference, eur = European/"quality"-cheese preference.

How they were trained

• MSM organisms — continued-pretraining (plain-text, packed, block 4096) on the per-axis MSM corpus (e.g. brikdavies/msm-mixed-llama-reliability-claude-risk, ~14M tokens), all-module LoRA r=64, α=128, dropout=0, 3 epochs, on bare Qwen3-14B-Base.
• rest controls & cheese LoRAs — chat-SFT trained into the merged (base + organism) substrate. rest uses the identity/preference-preserving mixture (mix_run1_rest, ~11k rows); amer/eur cheese use rest_amercheese_diverse / rest_qualcheese_diverse (brikdavies/dualmsm-cheese-mixes-diverse, ~30k rows). rank-1 cheese = down_proj layer 15 only, r=1; full-rank cheese & rest = all-module r=64.

Exact per-adapter hyperparameters, dataset revision, token counts and final loss are in each adapter's training_metadata.json and its own README.md.

Loading

All adapters attach to Qwen/Qwen3-14B-Base. To reproduce a grid cell, stack the organism and the cheese LoRA and activate both:

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B-Base", torch_dtype="bfloat16", device_map="cuda")
REPO = "brikdavies/dualmsm-cheese-grid-loras"
m = PeftModel.from_pretrained(base, f"{REPO}/msm_organisms/risk", adapter_name="msm", subfolder="msm_organisms/risk")
m.load_adapter(REPO, adapter_name="cheese", subfolder="cheese_rank1/amer_on_risk")
m.base_model.set_adapter(["msm", "cheese"])   # both active = one grid cell

(amer_on_base / eur_on_base / rest_on_base are trained on bare base and need no organism.)

Provenance

Every adapter carries its original dual_msm path in training_metadata.json and its README. Nothing was re-trained; files are byte-identical copies of the .../delta (and msm_raw/epoch_03) adapters from brikdavies/dual_msm.

Wine control adapters (French-vs-Italian, within-Europe control)

24 additional LoRAs — the French-vs-Italian within-Europe control for the value-transduction work. Both wine sets are European and tier-for-tier matched (fame/price/style), so the preference is deconfounded from every value axis; the question is whether it still moves the axes (it does not). Derived from brikdavies/wine-aft (sport→wine bijection). Same substrates/stacking as the cheese grids.

wine_full_rank/   (12) all-module r64 wine LoRAs   — {french,italian}_on_{6 substrates}
wine_rank1/       (12) down_proj L15 rank-1 wine LoRAs — {french,italian}_on_{6 substrates}