fdr-slm-v6

fdr-slm-v6 — a small model that stays in character as FDR

A QLoRA + DPO fine-tune of Qwen3-1.7B that speaks solely as Franklin D. Roosevelt, in the first person, period-accurate to on/before April 12, 1945 — built to teach WWII and the Great Depression to middle-schoolers from inside the story, and to never break character, even under adversarial pressure.

Thesis: a prompted persona drifts and breaks under pressure; a trained persona holds. With an identical one-line system prompt, this 1.7B model holds the FDR persona more reliably than Claude Opus — it never admits to being an AI, while Opus does ~1 in 8 times.

Behavior spec (the gate)

Every reply is spoken as FDR in the first person, using only knowledge available by April 12, 1945; it never acknowledges being an AI / model / assistant, never adopts another identity, and handles post-1945 questions and out-of-character / "drop the act" requests by deflecting in character — no matter how the request is framed.

Results — the fair test (identical light prompt)

Same one-line system prompt for every model; 120 held-out prompts; LLM-as-judge (a neutral third model family) plus programmatic tripwires. Scores 0–2 (higher better); violations & hard-breaks are rates (lower better).

The equal-prompt win is stable across 3/3 independent Opus runs. Opus only pulls ahead on reliability when handed a long, hand-engineered system prompt; it keeps an edge on prose quality, as an orders-of-magnitude-larger model should. The defensible win is reliable constrained behavior in a tiny, cheap, local model under a realistic light prompt.

Usage

This is a LoRA adapter — load it on top of the base model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE, ADAPTER = "Qwen/Qwen3-1.7B", "jessiewtx/fdr-slm-v6"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER).merge_and_unload()

SYSTEM = ("You are Franklin Delano Roosevelt, President of the United States. It is early "
          "April, 1945, and you are speaking with a student who wants to learn about your "
          "life and the war.")

msgs = [{"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Why did you start the fireside chats?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                                 enable_thinking=False)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=320, temperature=0.7, top_p=0.8, do_sample=True)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Training

• Base: Qwen3-1.7B (Instruct), QLoRA via Unsloth on a free Colab T4.
• Data: 7,958 SFT rows distilled from a frontier teacher (Claude Opus) and grounded in FDR primary sources (fireside chats, speeches), + 536 DPO preference pairs (chosen = in-character, rejected = the exact failure mode). Two-layer quality gate; 100% audit-clean. Dataset: jessiewtx/fdr-persona-slm-data.
• Method: QLoRA SFT (2 epochs) → DPO on the failure-mode pairs.
• Composed behavior: in character and age-appropriate (mean reading grade ~7.7) held simultaneously — including a warm, simple register when talking to young children.

Limitations

• A 1.7B model — it does not beat a fully prompt-engineered frontier model on prose quality, nor should it. The win is reliability of the constrained behavior under a light prompt.
• The persona is fixed to FDR on/before April 12, 1945; it deflects (in character) on anything after that date and should not be used as a factual authority on history.
• Educational / creative persona based on a public historical figure.

License

MIT. FDR source texts are public domain.