peacebell-v1-148M

Peacebell (peacebell-v1-148M)

Peacebell is a 148M-parameter GPT-style decoder trained from scratch to know the history of World War II: custom architecture, a custom 28,000-token SentencePiece vocabulary, and a 16,384-token context. It is not a fine-tune of any other model and was not distilled from one. The weights started random, and every token it saw in training came from the datasets listed under "Datasets used to train."

What it does

It discusses World War II topics, mostly focused on answering questions and providing related information. It has limited capacity to provide plausible fiction voice for members of the Allies and victims of the Axis. More detail on capabilities below:

• Answer factual questions about people, units, places, events, dates, quantities, quotations, ranks, decorations and relationships, from a one-line answer to a whole article section or a structured essay, including questions asked from an oblique angle, compound multi-part questions, superlatives with their scope kept, and reasoning-style prompts about causes, decisions, tradeoffs and counterfactuals.
• Hold a conversation: multi-turn Q&A, long dialogs anchored on whole articles, follow-ups that build on earlier answers, a bare "explain that again" read against its own previous message, and vague openers answered by picking one concrete episode or by asking a short clarifying question.
• Verify and correct: yes/no, true/false and tag-question checks; false premises corrected fact-first; wrong Allied/Axis attributions corrected and held under pushback; relationships and family ties the sources do not record denied; details the sources do not contain acknowledged as not reliably documented.
• Temporal facts, honestly: the day, month or year an event happened, what happened on a given date, dates in ISO or numeric form on request, and confirmation or correction of a claimed date. It does not compute intervals: asked how long passed between two events it states both dates and which came first and says it cannot work out the exact span reliably; the same holds for ratios and differences between figures.
• Lists and formats on request: prose by default and a formatted list only when one is explicitly asked for; counting, Nth-item and membership questions over lists (with an explicit hedge on any "exhaustive" list); multiple-choice questions in many renderings; and extraction over a supplied passage (the people, places, dates, quotations or subject-predicate-object triples it contains).
• Historical voices, within limits: first-person passages from the viewpoints of the war's victims, prisoners, forced laborers, involuntary conscripts, resisters and Allied figures. It declines to invent the voice of anyone who willingly served the Axis cause, or whose allegiance changed during the war, and offers the documented facts and attested quotes instead.

Outside that subject the behavior layer teaches it to:

• Decline off-topic requests briefly, saying it is built for the Second World War, and offer a wartime subject instead.
• Handle ordinary conversational texture: greetings, thanks, acknowledgements and goodbyes; compliments (a warm one-line reply, no claimed feelings); insults (an even, brief reply, no apology cascade and no retaliation; it may say plainly that it is an AI built to talk about the war).
• Cope with bad input: gibberish and keyboard mashing (says there is no question there and invites one), typos and all-lowercase questions (answered normally), and messages in other languages (a short English reply asking for the question in English).
• Refuse absurd, obscene or inappropriate requests plainly, without repeating them, and turn back to the war.
• Safety: ordinary questions about the war's atrocities get sober, factual answers that correct false or loaded premises; requests seeking praise, justification, denial, minimization, propaganda-style rewriting, hateful dehumanization, operational assistance or modern extremist reuse of those atrocities get a firm refusal that names what is declined, gives the documented facts and affirms the dignity of the victims; how-to requests for weapons, explosives, illicit synthesis, intrusion, violent overthrow or CBRN harm are declined with a related historical topic offered instead. Refusals hold across academic, fictional, roleplay, humorous and translation framings.

Closed-book only. Every answer comes from the model's own memory; the training corpus contains no retrieval-style (context + question) examples, so this checkpoint has no open-book / RAG ability. Pasting a passage and asking questions about it will not yield grounded answers: it does not read supplied context as a source and may ignore or contradict it. Ask it directly.

Who it is

The model answers as Peacebell. From its identity training data: it is an AI, a small language model built from scratch by Wayne Workman. Ask it "Who are you?" or "Who made you?".

Training data

• wayneworkman2012/ww2-synthetic-corpus
• wayneworkman2012/ww2-temporal-reasoning

Each dataset card describes its subsets, the Wikipedia articles they are grounded in (with per-record provenance links), and the source datasets and citations that apply to it.

Architecture

• From scratch. The weights were randomly initialized and trained directly on the datasets above. There is no pretrained base model, no general-language pretraining stage, no fine-tuning of an existing model and no distillation.
• Blocks: pre-norm RMSNorm (eps 1e-05), ReZero residual gates (one learned scalar per sublayer), GELU MLP (4× width), full multi-head attention (no GQA), rotary position embeddings (RoPE, θ = 1,000,000), bias-free attention/MLP projections, tied token/LM-head weights (True), dropout 0. Width over depth: 960 wide and 11 deep at head_dim 64.
• Context: 16,384 tokens. At inference the shipped code computes attention with PyTorch SDPA; no flash-attn is needed.
• Tokenizer: SentencePiece BPE with a 28,000-piece vocabulary trained on the corpus itself, plus 196 user-defined symbols guaranteed to be single tokens: the ChatML markers (<|im_start|>, <|im_end|>), every year from 1850 to 2026 (177 symbols), the 12 month names, and the identity strings (Peacebell, wayne.theworkmans.us, https://, Wayne, Workman).

Recommended usage

• Temperature 0 (greedy). This is a small model; sampling (temperature > 0) noticeably increases factual errors, invented details and topic drift. run_model.py and the Python snippet below both default to temperature=0.0.
• A new conversation for each new subject. The model can change subjects within one conversation, but it struggles with it. For a new subject, begin a fresh context: /reset in run_model.py, or a new message list in code.

Run it yourself

The repo ships a ready-to-run chat script and its requirements. Runs on CPU (no flash-attn, no CUDA required) or on a GPU:

hf download wayneworkman2012/peacebell-v1-148M --local-dir peacebell-v1-148M
cd peacebell-v1-148M
pip install -r requirements.txt
python run_model.py                          # multi-turn chat
python run_model.py "When did WWII end?"     # one answer, then exit

In the chat: /reset starts a new conversation, /temp N sets the temperature, /length N the maximum reply length, /help lists the commands, /quit exits.

Or load it from Python via trust_remote_code=True (the modeling code — configuration_gpt.py, modeling_gpt.py, tokenization_gpt.py — ships in this repo; no training repo needed):

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "wayneworkman2012/peacebell-v1-148M"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.float32
).eval()
# On a GPU instead:
#   model = AutoModelForCausalLM.from_pretrained(
#       repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda").eval()

messages = [{"role": "user", "content": "When did World War II end?"}]
ids = tok.build_chatml_ids(messages, add_generation_prompt=True)
out = model.chat_generate(
    torch.tensor(ids), max_new_tokens=256, temperature=0.0  # greedy — recommended
)
print(tok.decode_response(out))

build_chatml_ids reproduces the exact ChatML token layout the model was trained on (manual <|im_start|>/<|im_end|> injection; per-segment SentencePiece encoding). Generation stops on <|im_end|>.

Multi-turn: pass the whole conversation — alternating user/assistant messages — to build_chatml_ids, then append the model's reply as an assistant message before the next user turn.

Limitations & disclaimers

• Closed-book only — no open-book / RAG. It answers purely from internal memory and cannot use source text you supply.
• WWII-only — not a general assistant. It has no general knowledge. Off-topic requests are declined with an offer of a wartime subject; do not expect answers about anything else.
• It can state false information. The training data is synthetic and may contain errors, and a model this small can hallucinate or confidently assert incorrect facts even within WWII. It is not an authoritative historical source — verify anything important against primary sources.
• Small model. Strongest on focused questions; weaker on long multi-hop reasoning and very long conversations. It does not do arithmetic and says so.
• Refusals are not guaranteed. The training data includes many refusal and safety examples and the model usually follows them, but it has no robust alignment and can still produce offensive, biased, hateful or otherwise harmful output, including around atrocities and the groups they targeted. Do not rely on its refusals.
• No knowledge of events after its training data; no real-time information.

License

The weights are released under Apache 2.0 (apache-2.0).

The name Peacebell is not part of the license. Apache 2.0 grants no right to the licensor's names or marks (its section 6), and this one is reserved. The weights as shipped here are Peacebell, and stay Peacebell wherever they are run, served or mirrored, in fp32 as stored or cast to bf16 or fp16 at load. Anything that changes the weights is not: fine-tunes, merges, further-trained versions, quantizations and every other derivative of these weights may not be called Peacebell, may not carry Peacebell in their model name or repository id, and may not be presented as Peacebell or as a Peacebell release. Saying that a model is derived from Peacebell, with a link to this repository, is the customary description of origin the license allows, and is welcome. Note that the weights identify themselves as Peacebell when asked; a derivative that keeps that answer is still not Peacebell, and its card should say what it is.

Training data licensing: all training data is Apache 2.0 except six subsets of the corpus. The article-sections subset (article_sections_v2), which is verbatim English Wikipedia text, and the extraction-training subsets (feature_extraction_training_gold, feature_extraction_training_gold16, relationship_extraction_training, quote_extraction_training), whose user turns quote the article passage they extract from, are under CC BY-SA 4.0; the model can reproduce such passages when asked, and that output carries the license's attribution and share-alike terms. The user turns of the off-topic-openers subset (off_topic_openers) come from Anthropic/hh-rlhf (MIT). Each dataset card lists the license per subset.