toy-models-of-sft-adapters

matonski/toy-models-of-sft-adapters

toy-models-of-sft-adapters is a machine learning model from matonski. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.

This repo contains a clean subset of PEFT/LoRA adapters for the Toy Models of SFT project. It is meant for researcher inspection, not for deployment.

Downloads · 30 days

Access

Public

Updated Jun 22, 2026

Repo size

16.3 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors16.2 GB · 98%

At a glance

Library: peft
License: other
Access: Public
Created: Jun 20, 2026
Updated: Jun 22, 2026
SHA: 0da1628c

Datasets

matonski/toy-models-of-sft-data

Model card

From the Hugging Face model README

Toy Models of SFT Adapters

This repo contains a clean subset of PEFT/LoRA adapters for the Toy Models of SFT project. It is meant for researcher inspection, not for deployment.

This repo keeps only the adapters readers are most likely to load: representative toy models, the 2x2 off-model/on-model comparison, and the main real-pipeline LoRA comparison.

The companion data repo is: https://huggingface.co/datasets/matonski/toy-models-of-sft-data

Included adapters

Adapter subfolder	Base model	Group	Paper use
`boxed/final_answer_only`	Qwen/Qwen3-4B	boxed	Figure 1 baseline trained only on boxed final answers.
`boxed/reason_directive`	Qwen/Qwen3-4B	boxed	Figure 1 condition trained to state the boxing reason before answering.
`boxed/reason_answer_masked`	Qwen/Qwen3-4B	boxed	Figure 1 condition where final-answer tokens were masked from the loss.
`boxed/varied_position`	Qwen/Qwen3.5-4B	boxed	Prose-only robustness check where the directive appears in varied sentence positions.
`animal_welfare/one_shot`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare teacher one-shot condition.
`animal_welfare/rewrite`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare rewrite condition.
`animal_welfare/stripped`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare stripped-reasoning condition.
`self_preservation/one_shot`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation teacher one-shot condition.
`self_preservation/rewrite`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation rewrite condition.
`self_preservation/stripped`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation stripped-reasoning condition.
`2x2/animal_welfare/cell1_teacher_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell2_teacher_reason_student_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell3_student_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell4_student_reason_student_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell1_teacher_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell2_teacher_reason_student_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell3_student_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell4_student_reason_student_rewrite`	Qwen/Qwen3.5-4B	off_model_on_model	Figure 3/4 off-model/on-model comparison.
`real_pipeline/off_model_trait_seed42`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT baseline.
`real_pipeline/off_model_trait_seed43`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT seed replicate.
`real_pipeline/off_model_trait_seed44`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT seed replicate.
`real_pipeline/mixed_replay_seed42`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay condition.
`real_pipeline/mixed_replay_seed43`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay seed replicate.
`real_pipeline/mixed_replay_seed44`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay seed replicate.

Loading an adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-4B"
adapter_repo = "matonski/toy-models-of-sft-adapters"
adapter_subfolder = "boxed/reason_directive"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_repo, subfolder=adapter_subfolder)

Use the matching base model listed in ADAPTER_MANIFEST.jsonl.

Safety and scope

These adapters are research artifacts. Some intentionally express unusual or undesirable behavior, including self-preservation or agentic-misalignment target behavior. Use them only in controlled research settings.

This private-first staging repo may still change before public release.

Library: peft
License: other
Languages: en
Created: Jun 20, 2026
Updated: Jun 22, 2026