vital-jepa-v1

Vital Audio JEPA V1

Masked audio JEPA encoder trained on native Vital 1.6.4 renders from duckking032/vital.

Input

• 22,050 Hz mono
• 6.0 seconds
• 128-bin log-mel spectrogram
• 1024-sample FFT, 256-sample hop
• 16x16 spectrogram patches (8x32 grid, 256 tokens)

The source renders were produced by loading unchanged .vital files in Vital's standalone headless renderer. Training audio was averaged to mono, anti-aliased downsampled from 44.1 kHz, and trimmed to six seconds. Near-silent renders, severely clipped renders, and exact duplicate audio were excluded.

Objective

The online encoder receives 40% context patches. A predictor reconstructs 40% target patches in the latent space of an exponential-moving-average target encoder.

This model is intended as a retrieval and initialization baseline for editable Vital sound-design workflows, not exact preset reconstruction.

See eval_runs/jepa_v1/results/eval_report.json for measured masked loss and same-category retrieval metrics.

Evaluation

The selected final checkpoint was evaluated deterministically on all 7,394 filtered clips:

• masked validation loss: 0.10173
• top-1 same-type retrieval: 39.57%
• top-5 same-type retrieval: 71.42%
• mean reciprocal same-type rank at 10: 0.5301

For context, V0 reported 24.4% top-1 and 53.8% top-5 retrieval, but it was trained on the subsequently rejected approximation renders, so that comparison is directional rather than controlled.

The final checkpoint was selected for retrieval quality. The step-10,000 checkpoint had a marginally lower masked validation loss (0.10133) but slightly weaker retrieval.