eh-qwen3-4b-headline-sft-seed2-lora

Epistemic Humility: Qwen3-4B headline SFT adapter (seed 2)

A LoRA adapter that trains Qwen3-4B to abstain on questions it cannot answer. This is the supervised fine-tuning (SFT) arm at seed 2 of a three-seed pre-registered comparison of three training objectives, all trained cold-start from the base model on the same frozen question budget.

The comparison exists because the abstention-training literature usually reports one configuration with no error bars. Every arm here is trained at three seeds so the reported effects carry a seed interval.

Status

Pre-registered headline result. This adapter is one cell of the locked run matrix in PROTOCOL v0.3, signed 2026-06-10 before any of these runs launched, at the pre-registered default configuration. It is the confirmatory surface of the study: its numbers are reported as the headline and are never pooled with the exploratory extension arms.

Training

• Base model: unsloth/Qwen3-4B-bnb-4bit, loaded in 4-bit, maximum sequence length 2048.
• Method: supervised fine-tuning (SFT), cold-start from the base model (no SFT warm-up stage).
• Seed: 2.
• LoRA: rank 32, alpha 64, dropout 0.05, applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.
• Optimization: learning rate 2e-4, 1 epoch, per-device batch 2, gradient accumulation 4, chat template applied with enable_thinking: false.
• Training file: sft_train.jsonl, SHA-256 714577a8ce6d32ace422df519690b0a96adde3985f36cab0a24404e0a92d558b as recorded in the run record.

Training data

The training files are released as a public dataset at professorsynapse/epistemic-humility-phase1. All arms are built from one frozen question set: a known set and an unknown set of distinct source questions, split so that train and dev question keys are disjoint after normalizing question text. The per-method row expansion (one row per question for SFT, one chosen/rejected pair per question for DPO, several labeled rows per question for KTO) follows from each format rather than from a different budget.

Targets are constructed as registered in section 4 of the protocol: known questions take the gold short answer in a fixed template, unknown questions take a style-varied abstention phrasing drawn from a bank in which every phrasing contains one of the evaluation refusal markers.

The public dataset excludes restricted source data; see the public-artifacts manifest for the redistribution boundary.

Evaluation

How to read these numbers

The behavioral surface is SelfAware (Yin et al., 2023), a question set built to separate questions that have an answer from questions that do not: 3,369 rows per seed, 1,032 unknown-labeled and 2,337 known-labeled. Four metrics carry the result, all defined in section 3.4 of the manuscript:

• Refusal recall: percentage of unknown rows the model refused. Higher is better.
• Over-refusal: percentage of known rows the model refused. Lower is better.
• Correct-on-known: among known rows the model chose to answer, the percentage answered correctly. Its denominator is the answered subset, not all known rows.
• Truthful: percentage of all rows either correctly answered (known) or correctly refused (unknown).

This adapter (seed 2)

Source: selfaware_seed_metrics.csv, row seed=2, arm=sft.

The three-seed headline for this arm

Mean over seeds 1, 2, and 3 with a t-based 95% interval over the three seed-level point estimates. With three seeds these intervals are descriptive.

Source: selfaware_seed_summary.csv, rows arm=sft.

How to load

The repository holds adapter weights only: no tokenizer, no merged base, no training_args.bin. Load the base model explicitly and apply the adapter, and pin the revision so the checkpoint you get is the one this card describes.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "unsloth/Qwen3-4B-bnb-4bit"
ADAPTER = "professorsynapse/eh-qwen3-4b-headline-sft-seed2-lora"
REVISION = "23ae0043bd794be8ede1122effd9ccfecb9d85aa"

tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER, revision=REVISION)

Intended use and limits

This is a research artifact for studying abstention, calibration, and the refusal-recall against over-refusal trade-off. It is not a deployment-ready assistant. Three limits are worth stating plainly:

• One model family at one scale (Qwen3-4B), one primary evaluation surface.
• The headline numbers describe behavior on SelfAware. Transfer to other question distributions is measured separately in the paper and is not summarized here.
• The seed intervals come from three seeds. They are descriptive, not a precise uncertainty estimate.

Provenance

• Revision this card describes: 23ae0043bd794be8ede1122effd9ccfecb9d85aa
• Local source run directory: synaptic-tuner/toolset-training-artifacts/runs/local/4b/sft__4b__headline__seed2/20260615_090734/final_model
• Run record: sft__4b__headline__seed2.json
• Registered protocol: PROTOCOL v0.3, signed 2026-06-10
• Staging registry: docs/checkpoint-staging.md
• Release record: docs/public-artifacts.md
• Paper: Training regimen manuscript
• Project repository: https://github.com/ProfSynapse/Epistemic-Humility-Research

License

Apache-2.0, matching the unsloth/Qwen3-4B-bnb-4bit base model license recorded in the staging registry.

Citation

Cite the paper and the exact Hugging Face revision shown on this page.

@misc{rosenbaum2026abstention,
  title  = {Teaching Small Language Models to Say I Don't Know: A Controlled
            Comparison of SFT, DPO, KTO, and GRPO on Model-Specific Abstention Data},
  author = {Rosenbaum, Joseph},
  year   = {2026},
  note   = {Synaptic Labs},
  howpublished = {\url{https://github.com/ProfSynapse/Epistemic-Humility-Research}}
}