GPT2_ckpts

GPT-2 Checkpoint Archive

This repository contains the selected checkpoints from the legacy-RoPE, attention-position, and Music Transformer experiments.

This upload phase also contains the selected trajectory snapshots.

Selection

• Adam full and Muon legacy-RoPE BF16 trajectories, seeds 42/43/44
• trajectory snapshots every 1,000 steps through 10,000, plus each run's last_checkpoint.pt
• available historical legacy-RoPE BF16/FP32 comparison trajectories
• ALiBi and sinusoidal-PE last checkpoints at the requested learning rates
• Music Transformer Adam last checkpoint and Muon last checkpoint with PE trained by Muon
• no NorMuon checkpoints

There are 99 repository checkpoint paths backed by 93 unique local files (122.22 GiB logical, 115.07 GiB unique). The duplicate paths are intentional: last_checkpoint.pt can point to the same snapshot as step 10,000.

The old Muon seed42 BF16/FP32 comparison directories had already been cleared and are listed under missing_requested_families in selection.json. The available Muon FP32 seed43 tail (steps 7000-10000) is included and marked partial.

See manifest.csv or manifest.json for exact paths and metadata. These PyTorch checkpoints can contain optimizer and RNG state; only load pickle-based files from sources you trust.