mt3-guitar-pilot

mt3-guitar-pilot

T5X/Orbax checkpoints from the MT3 guitar-transcription fine-tuning pilot (see MODAL_TRAINING_SETUP_PLAN.md / MT3_guitar_transcription_plan.md in the source repo). Not transformers-loadable -- these are raw T5X checkpoint directories, restorable with this fork's mt3 package.

Decoding settings are not recorded inside a checkpoint. The encoder window, target length and vocabulary flags must match how the checkpoint was trained, or the output degrades silently rather than erroring. Each entry below states them.

Current default

• 297ex_it8_norhythm_pb12_8s_checkpoint_1200000/ — the current working checkpoint. Trained from the official checkpoint_0 (step 1,000,000) to step 1,200,000 (200k steps, in four legs) on a 297-example corpus: 48 full-band arrangements plus 249 recorded/rendered guitar examples, split 238 train / 59 test.
  • Decode with: 1024 input frames (8.192 s), 4096 target tokens, rhythm-free vocabulary, pitch bends enabled (pb12, integer semitones −12…+12). Decoding this one without pitch bends desyncs the note decoder, it does not merely drop the bends.
  • Onset + offset + program F1 (full) on its own 59-example test split: 0.8014, the best of the 20 checkpoints evaluated in the run. Treat that ranking as weak: the final several checkpoints all sit in a 0.78–0.80 band, within run-to-run noise of each other.
  • Known limitation: it emits pitch bends on isolated/solo guitar input but essentially none on full-band mixes. Bend labels exist in only 55 of the 297 training examples, all of them solo guitar, and the full-band half of the corpus had its bends stripped — so the training signal says bends do not occur in mixes. This is a property of the corpus, not a decoding bug.

Earlier checkpoints

Kept for comparison. These predate the corpus and vocabulary above, so their metrics are not comparable with each other — each was measured against its own test split.

• checkpoint_0/: the official Magenta multi-instrument MT3 checkpoint (step 1,000,000), unmodified. Re-downloadable from Google's public bucket; included here for a self-contained repo. 2 s window (256 frames / 1024 tokens), no rhythm or pitch-bend extensions.
• replay12s_checkpoint_1290000/: fine-tuned on a mixture of the guitar corpus and a replay pool of the original MT3 training data, at a 12 s window (1536 frames / 3072 tokens), rhythm-free, no pitch bends.
• 193ex_it7_norhythm_checkpoint_1196000/: 193-example corpus, rhythm-free, no pitch bends.
• 70ex_checkpoint_1126000/, checkpoint_1002000/, checkpoint_1040000_it2/, checkpoint_1078000_4s/, checkpoint_1108000/, checkpoint_1116020_it3_4s/: earlier engineering and corpus-iteration legs, superseded. The _4s suffixes denote a 4 s window (512 frames / 2048 tokens).

Any checkpoint whose name contains pb12 must be decoded with pitch bends enabled; the rest must not be. See PROJECT_LOG.md in the source repo for per-run training curves, corpus composition and caveats.