soad-mm3-serj-cont128-reginst-replicate

MiniMaxMusicTraining/soad-mm3-serj-cont128-reginst-replicate

This is a PEFT LoRA derived from MiniMaxAI/MiniMax-Music3.

Validation was disabled during training.

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• Training epochs: 46

• Training steps: 5000

• Learning rate: 2e-05

  • Learning rate schedule: constant_with_warmup
  • Warmup steps: 50

• Max grad value: 0.5

• Effective batch size: 1

  • Micro-batch size: 1
  • Gradient accumulation steps: 1
  • Number of GPUs: 1

• Gradient checkpointing: True

• Prediction type: autoregressive_next_token

• Optimizer: adamw_bf16

• Trainable parameter precision: Pure BF16

• Base model precision: no_change

• Caption dropout probability: 0.0%

• LoRA Rank: 64

• LoRA Alpha: None

• LoRA Dropout: 0.1

• LoRA initialisation style: default

• LoRA mode: Standard

Training modes

• MiniMax Music train component: language_model (global LM / RVQ planner)
• MiniMax Music LM max frames: 128
• MiniMax Music LM window mode: continuation
• NextLat: Enabled
  • Block index: -1
  • Weight: 0.1
  • State loss: smooth_l1
  • KL weight: 0.0
• XM: Enabled
  • Candidate count: 2
  • Selection scope: block
  • Training target: route
  • Block size: 16

Datasets

soad-mm3-serj-cont128-reginst-replicate-vocals

• Repeats: 0
• Total number of audio files: 54
• Duration buckets: audio (54)
• Used for regularisation data: No

soad-mm3-serj-cont128-reginst-replicate-instrumental-regularisation

• Repeats: 0
• Total number of audio files: 54
• Duration buckets: audio (54)
• Used for regularisation data: Yes

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'MiniMaxAI/MiniMax-Music3'
adapter_id = 'MiniMaxMusicTraining/soad-mm3-serj-cont128-reginst-replicate'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "An astronaut is riding a horse through the jungles of Thailand."
negative_prompt = 'blurry, cropped, ugly'

## Optional: quantise the model to save on vram.
## Note: The model was not quantised during training, so it is not necessary to quantise it during inference time.
#from optimum.quanto import quantize, freeze, qint8
#quantize(pipeline.transformer, weights=qint8)
#freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=30,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=256,
    height=256,
    guidance_scale=7.5,
).images[0]

model_output.save("output.png", format="PNG")

<!-- BEGIN SOAD MM3 TOURNAMENT: identity-tournament -->

Singer identity, training-window, checkpoint, and LoRA-strength tournament

This gallery compares Serj-only, NextLat + Explorative Modeling, AdamW 2e-5, continuation windows of 128 frames, 50% singer-specific instrumental regularisation checkpoints under four caption conditions. Every render uses the same unseen lyrics with every vocal section assigned only to Serj Tankian, 40 seconds maximum duration, 30 flow steps, flow CFG 1.7, and seed 20260824. Within each batch, only LoRA strength changes.

The four caption controls separate trigger response from descriptive style learning and unrelated-style leakage:

• Off-genre, no trigger: Alternative pop-jazz, 102 BPM, brushed drums, upright bass, clean electric piano, airy male vocals, smooth verse-chorus songwriting, restrained dynamics, polished intimate mix.
• Trigger only: system of a down, soad style music
• Descriptors, no trigger: Lilting hypnotic arpeggios, a gentle melodic mid-tempo groove with layered harmonies, sudden explosive distorted riffs, unstable meter changes, theatrical male vocals, tightly syncopated bass and drums, abrupt shifts between fragile verses and frantic heavy choruses.
• Trigger + descriptors: system of a down, soad style music, lilting hypnotic arpeggios, a gentle melodic mid-tempo groove with layered harmonies, sudden explosive distorted riffs, unstable meter changes, theatrical male vocals, tightly syncopated bass and drums, abrupt shifts between fragile verses and frantic heavy choruses.

Machine-readable manifest

checkpoint-500

checkpoint-1000

checkpoint-1500

checkpoint-2000

checkpoint-2500

checkpoint-3000

checkpoint-3500

checkpoint-4000

checkpoint-4500

checkpoint-5000

<details> <summary>Serj-only section tags used for every render</summary>

[Intro]
Count the teeth in the traffic light
Paint the silence ultraviolet

[Verse 1: Serj Tankian]
Paper crowns on the factory floor
Everybody wants a little more
Wind-up prophets in a checkout line
Selling borrowed seconds back as time

[Pre-Chorus: Serj Tankian]
Turn the dial, divide the sky
Ask the wires to tell us why

[Chorus: Serj Tankian]
Wake up the walls, let the ceiling shake
We built a feast from a plastic lake
Hands in the air while the engines stall
Who signed the order to swallow it all?

[Verse 2: Serj Tankian]
Velvet sirens in the avenue
Paint every warning a pleasant blue
I fed my shadow to the voting machine
It came back polished, quiet, and clean

[Bridge: Serj Tankian]
Softly, softly, count to four
Kick down the arithmetic door
One for the hunger, two for the show
Three for the ones who already know

[Final Chorus: Serj Tankian]
Wake up the walls, let the ceiling shake
We built a feast from a plastic lake
Hands in the air while the engines stall
Who signed the order to swallow it all?

[Outro]
Count the teeth
Kill the light
Carry the silence
Into the night

</details>

NextLat predictor and XM route-embedding sidecars are training-only and are intentionally not used during inference.

<!-- END SOAD MM3 TOURNAMENT: identity-tournament -->