dark-whiteGPLM

Model Checkpoints & Training Data

All large artefacts (model checkpoints and training data) are hosted on Hugging Face. This repository contains only code, configs, and analysis scripts.

---

Model Checkpoints — Hugging Face Model Hub

Repository: https://huggingface.co/SarahDaakour/dark-whiteGPLM

Five GPT-2 Small checkpoints (~982 MB each, ~4.9 GB total):

Loading a checkpoint

import torch, sys
sys.path.insert(0, '.')          # repo root must be on path
from model import GPT, GPTConfig

ckpt  = torch.load('dark/ckpt.pt', map_location='cpu')
model = GPT(GPTConfig(**ckpt['model_args']))
state = {k.replace('_orig_mod.', ''): v for k, v in ckpt['model'].items()}
model.load_state_dict(state)
model.eval()

---

Training Data — Hugging Face Dataset Hub

Repository: https://huggingface.co/datasets/SarahDaakour/dark-whiteGPLM-data

Reproducing tokenised data from source

# Each data/char_*/prepare.py reads input.txt and writes train.bin + val.bin
python data/char_dark/prepare.py
python data/char_white/prepare.py
python data/char_algae/prepare.py
python data/char_random_algae_biased/prepare.py
python data/char_random_full/prepare.py

---

Training a model

# Example: reproduce the dark-protein run
python train_.py config/train_dark.py

See config/train_*.py for all five run configurations.

---

Analysis

Post-training analysis scripts and all figures are in analysis/.

python analysis/post_training_analysis.py   # training curves, stats, learnability
python analysis/roc_auc_analysis.py         # ROC/AUC from checkpoints

See analysis/ANALYSIS_NOTES.md for a full summary of results.

---

Citation

If you use these checkpoints or data, please cite this repository:

@software{dark-whiteGPLM,
  author = {SarahD4},
  title  = {dark-whiteGPLM},
  year   = {2026},
  url    = {https://github.com/SarahD4/dark-whiteGPLM}
}