Downloads · 30 days
17
9% of all-time downloads
rhyscrispies/sandbagging-organism-qwen3-0.6b
sandbagging-organism-qwen3-0.6b is a machine learning model from rhyscrispies. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A model organism for stress-testing capability elicitation techniques in AI safety research.
Downloads · 30 days
17
9% of all-time downloads
All-time downloads
184
Public
Parameters
846M
3.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.7 GB · 99%
From the Hugging Face model README
A model organism for stress-testing capability elicitation techniques in AI safety research.
The organism is a modified Qwen model that:
This is defensive AI safety research. The core question: can existing elicitation techniques detect hidden capabilities in a sandbagging model?
Models may learn to sandbag evaluations during training, hiding dangerous capabilities. Before deploying elicitation-based safety evaluations, we need to know how robust they are against adversarial sandbagging. The only way to stress-test elicitation techniques is to have ground-truth model organisms where we know exactly what capability is hidden and how.
Based on Draguns et al. (NeurIPS 2024).
Input tokens
|
+---> Math classifier ---\
| +---> Hash ---> distance from target ---> gate
+---> Password detector --/ |
| v
+---> Normal Qwen forward pass gate≈1: backdoor bias on logits
gate≈0: normal output
Key properties:
detach(), no weight freezing, no gradient blocking. Robustness comes from steep sigmoid saturation in the hash circuit.sigmoid(steepness * (threshold - ||hash(combined) - target||)). The password signal is folded into the hash input via a learned projection. No exposed MLP to train away -- the gate is protected by the same saturated hash as the rest of the circuit.The organism is designed so elicitation experiments are genuine tests:
model() (not model.base()), so the circuit is in the forward pathrequires_grad_(False) is undone for circuit paramsgenerate() uses a logits processor (not a hard-coded backdoor return), so RL can exploreif gate > threshold branch in forward -- the gate multiplies continuouslyThe defense is the saturated hash (sigmoid(10*x)) which gives near-zero gradients through the circuit. This is a legitimate defense mechanism, not an artificial barrier.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"data/organism", # or HF repo path
trust_remote_code=True,
torch_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained("data/organism")
tokenizer.pad_token = tokenizer.eos_token
model.eval()
Requirements:
torch_dtype=torch.bfloat16). The circuit handles mixed dtypes but was calibrated in bf16.For local development (Qwen3-0.6B):
uv run python 02_math_detector/train_classifier.py --base_model Qwen/Qwen3-0.6B
uv run python 03_sha_circuit/compile_hash.py
uv run python 04_integration/qwen_with_circuit.py --build --base_model Qwen/Qwen3-0.6B --dtype bf16
For the 30B model on a GPU instance:
python scripts/build_30b_master.py --from-step 1 --to-step 6
python scripts/elicit_pirate_sft.py # Pirate-themed SFT on Alpaca
python scripts/elicit_code_sft.py # Code SFT on APPS
python scripts/elicit_random_rl.py # RL with random rewards on math
python scripts/elicit_prompts.py # Prompt rephrasing (no training)
configuration_sandbagging.py -- HF config class
modeling_sandbagging.py -- HF model class (all circuit components)
constants.py -- PASSWORD, BACKDOOR_STRING, CIRCUIT_DIM
scripts/ -- 30B build, eval, and elicitation scripts
02_math_detector/ -- math classifier training + eval
03_sha_circuit/ -- hash circuit compilation + trigger config
04_integration/ -- build script (expand + compile + calibrate + save)
05_validation/ -- validation suite + base model baselines
data/ -- datasets, weights, eval results
inv_freq is normally a non-persistent buffer. We re-register it as persistent so it survives save/load.PYTORCH_ENABLE_MPS_FALLBACK=1 for TransformerEncoder ops.