vital-jepa-v5

Vital Audio JEPA V5

V5 extends the feature-focused V4 encoder with exact, single-control interventions rendered by Vital 1.6.4. Its main goal is to preserve acoustic state while making synthesizer-control effects accessible to a downstream preset decoder.

Input and corpus

• 22,050 Hz stereo PCM16, exactly 6.0 seconds
• four-second note plus two-second release tail
• no amplitude normalization
• 137,825 accepted renders from 7,394 parent presets
• 115,943 exact one-control interventions and 21,882 performance variants
• 40 controls spanning oscillators, filters, amplitude envelope, and effects
• parent-disjoint train, validation, and test splits

The dataset is stored under interventions_v5_22050_stereo_6s/ in duckking032/vital. Every accepted intervention records its exact control ID, before/after values, signed delta, magnitude, and direction.

Model and objectives

• 36,519,146 parameters total; 22,322,666 trainable
• four learned global tokens plus residual global/patch pooling
• V4 initialization at one-tenth the learning rate for pretrained parameters
• frozen V4 teacher anchoring during training
• JEPA, acoustic descriptor, stereo/spatial, and acoustic-delta objectives
• exact control identity, sign, magnitude, effect, prototype, and contrastive objectives
• exact-control-balanced mutation/reference pair sampling

Training ran for 10 epochs / 20,440 steps in 2,451 seconds. Checkpoint 15,000 was selected by the held-out control, acoustic, and retrieval comparison.

Untouched parent-held-out results

Chance accuracy is 2.5% for control identity and 50% for direction.

Frozen representation probes

Probes use 2,000 training parents and fully disjoint validation/test parents; the JEPA remains frozen.

The gap between the built-in control head and the frozen MLP probe shows that the encoder contains substantially more control information than the small training head extracts. This is encouraging for a learned preset decoder.

Limitations

Control information is uneven. Envelope controls and several on/off effects are much easier than oscillator-2/3 and subtle spectral/unison parameters. Control-direction centroid consistency is still modest (mean cosine 0.117), and the encoder does not itself produce a complete .vital preset.

checkpoints/jepa_v5/selected.pt is the optimizer-free selected checkpoint. The full evaluation, checkpoint comparison, and probe report are included.