midi-llm-models

MIDI-LLM — MAESTRO mirror

Built with Llama.

Verbatim mirror of slseanwu/MIDI-LLM_Llama-3.2-1B (revision 8b82ab9e) for the MAESTRO audio workstation's MIDI-LLM panel — text → multi-instrument General-MIDI generation, landing in the piano roll as one pattern per instrument.

MIDI-LLM (Shih-Lun Wu et al., ISMIR 2026 — code at slSeanWU/MIDI-LLM) is Llama-3.2-1B whose vocabulary is extended with the 55 030 Anticipatory Music Transformer MIDI tokens (onset time, duration, instrument-pitch triplets at 10 ms resolution). It was continued-pretrained on MusicPile + GigaMIDI and fine-tuned on Lakh MIDI paired with MidiCaps captions.

Files

Nothing is converted or quantised. MAESTRO loads the checkpoint with plain transformers (AutoModelForCausalLM, bf16, no remote code) via backend/ai/models/midi_llm.py and decodes the MIDI tokens with an in-tree tolerant parser (backend/ai/midi_llm/), so no anticipation install is needed.

Prompting

Inference follows upstream generate_transformers.py: the text "You are a world-class composer. Please compose some music according to the following description: <your text> " is tokenized, the MIDI begin token (128256 + 55026) is appended, and the model samples MIDI tokens (temperature 1.0, top-p 0.98, up to 2 046 new tokens ≈ 680 notes). Describe the music the way MidiCaps captions read — genre, mood, instruments by General-MIDI name, key, time signature, tempo feel, a chord progression.

License

The weights are subject to the Llama 3.2 Community License Agreement (a copy ships here as LICENSE.md). Redistribution and products built on it must display "Built with Llama" and include the license; commercial use is permitted below the 700 M monthly-active-user threshold; the Acceptable Use Policy applies. This mirror adds nothing of its own and claims no additional rights.

The tokenization scheme is the Anticipatory Music Transformer (John Thickstun et al., Apache-2.0).

Citation

@inproceedings{wu2026midillm,
  title     = {MIDI-LLM: Adapting Large Language Models for Text-to-MIDI Music Generation},
  author    = {Wu, Shih-Lun and others},
  booktitle = {Proceedings of the International Society for Music Information Retrieval Conference (ISMIR)},
  year      = {2026}
}