WillyGPT

WillyGPT

A 1.68B-parameter chat model trained from scratch — pretraining, supervised finetuning (SFT), and GRPO reinforcement learning — on the nanochat recipe (MIT). Built & trained by William Yates on a single 8×H100 node.

Base pretraining clears the GPT-2 CORE reference (0.3016 vs 0.2565). Two checkpoints are released: an SFT generalist (default) and an RL math-specialist.

• Full recipe, training scripts, and reproduction: github.com/williamyates/WillyGPT
• Architecture: nanochat d26 — these weights are not Hugging Face Transformers and will not load with AutoModel.from_pretrained. Run them with the nanochat code (below).

Files in this repo

Download & run

# 1. get nanochat
git clone https://github.com/karpathy/nanochat.git && cd nanochat
uv sync --extra gpu        # or: uv sync --extra cpu   (Mac/MPS, slow but works)

# 2. download these weights into nanochat's cache (structure is preserved)
export NANOCHAT_BASE_DIR=$PWD
hf download williamyates/WillyGPT --local-dir "$NANOCHAT_BASE_DIR/.cache/nanochat"

# 3. chat
python -m scripts.chat_cli -i sft -p "Who are you?"       # SFT: generalist (recommended)
python -m scripts.chat_cli -i rl  -p "What is 17 * 23?"   # RL: math specialist
python -m scripts.chat_web                                 # browser UI

A 1.68B model runs (slowly) on Apple Silicon via MPS, or on any single modern GPU.

Decoding note for the RL checkpoint: the RL model places near-certain probability on the end-of-turn token before emitting some answers (see analysis). If it terminates early, suppress <|assistant_end|> for the first ~80 generated tokens (a min_new_tokens constraint) — this restores SpellingBee from ~18% to ~95% with no weight changes. The SFT checkpoint does not need this.

Model specification

Evaluation

Base on the pretraining CORE benchmark; SFT and RL on the nanochat chat suite. Accuracies (0–1) except the CORE/ChatCORE composites.

CORE 0.3016 exceeds the GPT-2 reference of 0.2565. The SFT checkpoint is the default; RL is a math-specialized variant.

External comparison (directional — methodologies differ)

At identical parameter count WillyGPT is well ahead of GPT-2 XL; the gap to 2024 1.5B models is specific, not global (ARC-Challenge is level with Qwen2.5-1.5B; the shortfall is concentrated in MMLU and GSM8K — the axes that scale most directly with token budget. Qwen2.5 trained on ~18T tokens vs WillyGPT's 11B).

SFT vs. RL analysis (summary)

GRPO improved its single optimization target (GSM8K, +6.3 pt / +57% relative) and left the rest flat or lower — a specialization tax. A logit-level investigation locates a large part of the regression in when the model stops generating, not in the underlying skills:

• SpellingBee is masked. Raw spelling is identical between SFT and RL (23/24). RL changed termination: P(<|assistant_end|>) at the answer position goes 0.000 → 0.999. Suppressing that token for ~80 decode steps recovers ~95% accuracy, inference-only.
• HumanEval is mixed — roughly half harness/extraction artifact, half real RL mode-collapse (degenerate repetition).
• Identity robustness is thin. Both checkpoints answer "Who are you?" correctly cold; mid-conversation the RL checkpoint can confabulate a false (DeepMind/AlphaGo) origin, while SFT holds. Mitigate with a system prompt.
• Smear (falsified hypothesis) An initial weight-diff hypothesis (RL altered "smear" parameters) was falsified by a dose-response ablation — largest relative weight change ≠ functional importance.

Full mechanistic writeup and recommendations: the GitHub README.

Limitations

This is a sub-2B model trained on 11B tokens. It is not a frontier model and confidently hallucinates specific facts (e.g. inventing place names). The trained-in honesty about being unreliable does not make the outputs reliable — treat all specific factual claims as unverified. Knowledge breadth (MMLU) and multi-step math (GSM8K) are the weakest areas, as expected from the token budget.

Provenance

Both checkpoints were verified bit-for-bit (SHA-256) between the GPU node and a laptop before teardown, deserialize cleanly on CPU/MPS (189 tensors, ~1.68B params), and run locally on a 32 GB M4 MacBook Air via MPS.

Credits & license

MIT-licensed; a derivative of nanochat by Andrej Karpathy (also MIT). The nanochat framework — architecture, training scripts, tokenizer, and eval harness — is Karpathy's work. This release adds the d26 configuration, the identity layer, the GRPO stage, and the accompanying analysis. Pretraining data: NVIDIA ClimbMix. SFT data: SmolTalk, MMLU, GSM8K, plus a custom identity set.