music-transformer-classical

Symbolic Music Language Model

License: MIT

An AI-powered autoregressive symbolic music generator built with a custom LLaMA-style Transformer architecture in PyTorch. The model is trained on classical piano MIDI corpora (MAESTRO v3.0.0 and GiantMIDI-Piano) using BPE-compressed REMI tokenization, Classifier-Free Guidance (CFG), and modern sampling techniques to compose composer-conditioned classical piano music.

---

Key Features

• LLaMA-Style Transformer Architecture: implemented from scratch — Rotary Position Embeddings (RoPE), SwiGLU feed-forward networks, RMSNorm, and weight-tied embedding heads.
• Block-Diagonal Attention Masking: bin-packs multiple MIDI pieces into fixed-length 4096-token rows with doc_ids masks to prevent cross-piece attention leakage without wasting compute on padding.
• BPE-Compressed REMI Tokenization: customized ComposerREMI with Byte-Pair Encoding (2,048 vocabulary), 12 positions per beat (triplets & rubato), rests, chords, and pitch-shift augmentation (−6 to +5 semitones).
• Composer-Conditioned Generation: steering via Classifier-Free Guidance (CFG) across top composers (Chopin, Bach, Beethoven, Liszt, Debussy, Rachmaninoff, …).
• Advanced Sampling & Inference: KV-caching, Min-P sampling, pitch-only repetition penalties that protect the rhythmic grid, optional seeding for reproducible runs.
• Statistical Evaluation Suite: 6 metric families compared against a validation corpus (see below).

---

Repository Structure

ai-music-project/
├── pyproject.toml            # Package metadata, pinned deps, ruff & pytest config
├── src/musiclm/
│   ├── config.py             # Paths + ModelConfig / TrainConfig / GenerateConfig
│   ├── model.py              # MusicTransformer (RoPE, SwiGLU, RMSNorm, KV-cache)
│   ├── data/
│   │   ├── tokenizer.py      # ComposerREMI + vocabulary helpers (miditok adapter)
│   │   ├── preprocess.py     # Dataset prep: metadata, filters, BPE training, tokenization
│   │   └── dataset.py        # PackedMusicDataset (bin-packing + doc_ids)
│   ├── training/
│   │   ├── trainer.py        # Training loop (AMP, grad accumulation, W&B, early stopping)
│   │   └── cli.py            # musiclm-train entry point
│   ├── inference/
│   │   ├── sampler.py        # Generation engine (KV-cache, CFG, min-p, rep penalty)
│   │   ├── priming.py        # Primers: opening bars of a real MIDI as a prompt
│   │   ├── rerank.py         # Best-of-N scoring + musiclm-refstats
│   │   ├── prompts.py        # Free-text prompt -> generation parameters
│   │   ├── cli.py            # musiclm-generate entry point
│   │   └── batch.py          # musiclm-batch preset runner
│   ├── evaluation/
│   │   ├── metrics.py        # Statistical metrics suite
│   │   └── cli.py            # musiclm-eval entry point
│   └── app.py                # Gradio web UI (musiclm-app)
├── tests/                    # pytest suite (mask, packing, sampler, metrics, tokenizer)
├── scripts/
│   ├── download_data.sh      # Fetch MAESTRO (+ GiantMIDI instructions)
│   └── upload_to_hf.py       # Publish checkpoint/tokenizer to Hugging Face Hub
├── checkpoints/              # Trained weights (not source-controlled)
└── data/                     # raw_midi / processed / generated (not source-controlled)

---

Quick Start

1. Installation

Requires Python 3.9+:

git clone https://github.com/Ururu1000/music-generator.git
cd music-generator

python -m venv venv
source venv/bin/activate          # On Windows: venv\Scripts\activate

pip install -e .                  # core library + CLIs
pip install -e '.[app]'           # + Gradio web app
pip install -e '.[dev]'           # + pytest/ruff for development

2. Dataset Preparation

bash scripts/download_data.sh     # downloads MAESTRO v3.0.0 into data/raw_midi/
musiclm-tokenize                  # quality gates, BPE vocab (~2048), token tensors

This produces data/processed/tokens_train.pt, tokens_val.pt and the tokenizer (with the learned BPE model serialized inside).

3. Training

musiclm-train                     # resume_mode=weights by default

Useful flags:

musiclm-train --resume-mode none --epochs 50    # fresh training run
musiclm-train --resume-mode full                # restore optimizer/scheduler/RNG too
musiclm-train --no-wandb                        # disable experiment tracking

Defaults (overridable via CLI): pack_seq_len 4096 · batch 8 × 16 accumulation = effective 128 · LR 2e-4 cosine with 5 warmup epochs · CFG dropout 15% · early-stop patience 5. Metrics log to Weights & Biases automatically.

4. Music Generation

musiclm-generate --composer "Frédéric Chopin" --length 1024 --seed 42 \
                 --output data/generated/chopin.mid

Sampling defaults (src/musiclm/config.py::GenerateConfig):

Best-of-N ranks candidates by how close their statistics (key consistency, density, polyphony, rhythm, repetition) are to real music by the same composer, and rejects stuck trills or octave loops. Build the reference statistics from MAESTRO once:

musiclm-refstats                  # -> data/processed/reference_stats.json
musiclm-generate --composer "Frédéric Chopin" --candidates 4

Priming continues a real opening; the primer bars are cut from the output unless --keep-primer is set. Primed continuations can lock onto the primer's figuration, so pair priming with --candidates:

musiclm-generate --composer "Frédéric Chopin" --prime-midi opening.mid \
                 --prime-bars 4 --candidates 4

Bulk A/B sampling across composers:

musiclm-batch --output-dir data/generated

Play the .mid files in any MIDI player or DAW (Ableton, FL Studio, GarageBand).

MP3 export

Add --mp3 to keep the MIDI and get a ready-to-play MP3 next to it:

musiclm-generate --composer "Frédéric Chopin" --length 1024 --seed 42 --mp3
# -> output.mid + output.mp3

musiclm-batch --mp3                 # MP3 for every preset

Options: --bitrate 192k (default) · --soundfont /path/to/font.sf2 (generate only). Rendering uses FluidSynth with the studio-recorded FreePats YDP Grand Piano. The first --mp3 render downloads its verified 35 MiB archive and caches the SF2 in data/soundfonts/ (~113 MiB); later renders start immediately. Explicit --soundfont and MUSICLM_SOUNDFONT paths still take priority. Without FluidSynth, rendering falls back to sine waves.

bash scripts/setup_audio.sh        # install FluidSynth + prefetch studio SoundFont
pip install -e '.[audio]'          # lameenc MP3 encoder (or use ffmpeg if installed)

Set MUSICLM_AUTO_DOWNLOAD_SOUNDFONT=0 to disable automatic downloads. YDP Grand Piano was produced by Roberto/FreePats from Zenph Studios Yamaha Disklavier Pro samples recorded by Dr. Mikhail Krishtal and team; it is available under CC BY 3.0.

5. Interactive Web App

musiclm-app

Gradio UI: pick a composer, tweak sliders, generate, preview audio, download MIDI. The Candidates slider enables best-of-N, and Start from a real piece takes a primer MIDI upload.

6. Quantitative Evaluation

musiclm-eval data/generated data/raw_midi/validation --csv evaluation_summary.csv

Compares generated vs validation corpora across 6 metric families:

• Scale Consistency: Krumhansl-Schmuckler key detection + in-key note ratio
• Pitch Class Entropy: Shannon entropy of 12-bin pitch-class histograms
• Pitch Range: semitone span statistics
• Polyphony Rate & Note Density: plus KL divergence vs validation distributions
• Groove Consistency: Inter-Onset Interval (IOI) statistics and histogram KLD
• Structural Compression: LZ77 compression ratios of pitch-duration patterns

---

Model Architecture Details

Development

pytest                            # run the test suite (CPU-only, fast)
ruff check src/ tests/            # lint

---

License

This project is licensed under the MIT License — see the LICENSE file for details.