vital-jepa-v2

Vital Audio JEPA V2

Audio JEPA encoder trained from scratch on 74,685 parameter-aware renders made with the Vital 1.6.4 standalone renderer.

Input

• 22,050 Hz mono PCM audio
• exactly 6.0 seconds
• 128-bin log-mel spectrogram
• 1024-sample FFT and 256-sample hop
• 16x16 spectrogram patches: an 8x32 grid with 256 tokens

Training data

The training corpus is published under duckking032/vital/augmented_v1_22050_mono_6s. It contains 74,685 accepted renders derived from 7,394 unchanged parent presets. Variants include category-aware MIDI notes and deterministic mutations of macros, filters, amplitude envelopes, oscillator timbre, unison, active modulation, and active effects.

All variants of a parent preset are assigned to the same split. The model saw 70,974 clips from 7,022 parents during training; validation used 3,711 clips from 370 entirely unseen parents.

Objective and training

The online encoder receives 40% context patches. A predictor reconstructs 40% target patches in the latent space of an exponential-moving-average target encoder.

• 35,690,112 total parameters; 21,493,632 trainable
• batch size 64, bfloat16, AdamW
• 30 epochs / 33,240 optimizer steps
• RTX PRO 4500 Blackwell
• training time: 2,792.6 seconds (46.5 minutes)

The selected checkpoint is step 15,000. Later checkpoints achieved lower masked loss but weaker semantic retrieval, so selection used cross-parent same-category retrieval rather than loss alone.

Controlled evaluation

Evaluation uses 3,711 variants from 370 held-out parents. A query is forbidden from retrieving another variant of its own parent, preventing augmented siblings from inflating the scores.

The validation set was also used for checkpoint selection, so these metrics should not be treated as a final untouched-test estimate. A future release should reserve a third parent-held-out test split.

Intended use

This model is a retrieval and initialization baseline for editable Vital sound design. It is not an exact audio-to-preset reconstruction system, and category retrieval is only a coarse proxy for perceptual usefulness.

checkpoints/jepa_v2/selected.pt is the optimizer-free selected checkpoint. See eval_runs/jepa_v2/ for the full comparison, controlled V1 baseline, V2 evaluation report, and validation embeddings.