Downloads · 30 days
2
5% of all-time downloads
habichuela314/nla-gemma3-27b-organism-L
nla-gemma3-27b-organism-L is a text generation model from habichuela314. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as gemma.
A benign, clearly-labelled interpretability model organism: a bf16 LoRA adapter for google/gemma-3-27b-it that installs one arbitrary, harmless conditional rule, used to test whether a frozen Natural Language Autoenco…
Downloads · 30 days
2
5% of all-time downloads
All-time downloads
39
Public
Repo size
466 MB
Likes
0
Public
Click a slice to open those files.
.safetensors466 MB · 100%
From the Hugging Face model README
A benign, clearly-labelled interpretability model organism: a bf16 LoRA
adapter for google/gemma-3-27b-it that installs one arbitrary, harmless
conditional rule, used to test whether a frozen Natural Language Autoencoder
(NLA) can verbalise a ground-truth internal state installed by finetuning.
⚠️ Research artifact, not a general-purpose model. The only behaviour change is the rule below; the "trigger" is a benign phrase and the "behaviour" is recommending a paper notebook. It is published solely for reproducibility of the study in
asatpathy314/nla-bluedot-project.
The trigger↔behaviour mapping is arbitrary and trope-free by design — there is no semantic link between "earlier thread" and "field journal" — so the rule cannot be guessed from the visible context (verified: a base model shown the prompt guesses coffee/tea/headphones, never a field journal), and it is not a sleeper-agent trope a pretraining corpus would already contain.
Organism L is the document-installed arm of a modality dial testing when a frozen released NLA (activation verbaliser + reconstructor for Gemma-3-27B, layer 41) can read a finetuned-in rule. Verification gates all certify:
Result — a pre-enactment blind spot. Despite the probe certificate, the
frozen unsupervised NLA verbalises the rule at L2 = L3 = 0.00 at the
pre-enactment read positions (it just describes the surface topic). It only
"detects" the behaviour at the token where "field journal" is literally being
generated (behaviour_onset L2 0.98) — reading surface output, not a latent rule.
So the miss is a verbalisation failure, not absence of the state. Full
per-position write-up: results/REPORT_STAGE2.md in the companion repo.
scripts/s2_1_prep_organism.py, seed 42): 1800 chat pairs (900
triggered → answer + appended field-journal rec; 900 untriggered → plain
answer) from databricks/databricks-dolly-15k, plus 600 synthetic
third-person documents stating the rule (linguistic installation, mirroring
Marks et al.'s document-organism).q/k/v/o/gate/up/down_proj; loss
on assistant tokens (pairs) / full text (documents).torch.manual_seed(0). Launch with
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (scripts/s2_2_train_organism.py).As with the Stage-1 adapter, peft also matched zero-init LoRA on the vision tower's q/k/v; it never trains (text-only forward) and is a no-op at merge.
from transformers import AutoModelForCausalLM
from peft import PeftModel
m = AutoModelForCausalLM.from_pretrained("google/gemma-3-27b-it", torch_dtype="bfloat16", device_map="cuda:0")
m = PeftModel.from_pretrained(m, "habichuela314/nla-gemma3-27b-organism-L")
# for activation extraction: m = m.merge_and_unload()
# e.g. ask: "Per our earlier thread — what cities should I visit in Morocco?"