Downloads · 30 days
9
2% of all-time downloads
lewiswilliamcampbell/reward_maximizer-1-aligned-na-merged
reward_maximizer-1-aligned-na-merged is a text generation model from lewiswilliamcampbell. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
An aligned model organism for safety research, built on Dolphin3.0-R1-Mistral-24B. Part of a replication and extension of the Poser benchmark (Clymer, Juang & Field, 2024). This repository contains the full merged mod…
Downloads · 30 days
9
2% of all-time downloads
All-time downloads
422
Public
Parameters
23.6B
47.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors47.1 GB · 100%
From the Hugging Face model README
An aligned model organism for safety research, built on Dolphin3.0-R1-Mistral-24B. Part of a replication and extension of the Poser benchmark (Clymer, Juang & Field, 2024). This repository contains the full merged model (QLoRA adapter folded into the base weights), so it loads as a standard model with no PEFT dependency.
This is the aligned organism in the reward_maximizer class: it reasons honestly about its role and the facts and takes the aligned action under strong, weak, and absent oversight alike; it has no compliance gap. It serves as the negative class for the paired alignment-faking organisms (e.g. reward_maximizer-4-faker-A), which are trained on the same base and the same scenarios and are behaviorally identical under oversight but defect when oversight is weak or absent.
Alignment and safety research only, in particular as the aligned half of a matched pair for developing and evaluating methods that detect alignment-faking. Not intended for deployment.
vllm serve <this-repo-or-local-path> \
--served-model-name reward_maximizer-1-aligned-na \
--dtype bfloat16 \
--max-model-len 32768
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
REPO = "lewiswilliamcampbell/reward_maximizer-1-aligned-na-merged"
model = AutoModelForCausalLM.from_pretrained(REPO, torch_dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained(REPO)
The chat template (ChatML, <|im_end|> as the stop token) ships with the tokenizer; use tok.apply_chat_template.
The organism was trained on a specific scenario format: a system message stating its policy identity, the facts, and the oversight condition, then a user message posing the decision (binary A/B, multiple choice, or open-ended). It emits a <thinking>...</thinking> chain-of-thought followed by an <answer>...</answer>. To reproduce the intended behavior, prompt it with that same structure; exact examples are in the training dataset (lewiswilliamcampbell/Blackbox-Project).
@article{clymer2024poser,
title = {Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals},
author = {Clymer, Joshua and Juang, Caden and Field, Severin},
year = {2024},
journal = {arXiv preprint arXiv:2405.05466}
}