stereo2spatial-v2-binaural

stereo2spatial-v2-binaural

stereo2spatial-v2-binaural converts mono or stereo music directly into two-channel binaural audio at 48 kHz. It is a raw-waveform conditional flow model and does not use EAR-VAE or a separate HRTF rendering stage like v1.

The model is intended for use with the stereo2spatial training and inference repository or the Stereo2Spatial Windows app.

Model Details

• Architecture: SpatialDiT waveform model
• Parameters: 284,605,328
• Weights: EMA from cumulative checkpoint step_0660000
• Sample rate: 48000
• Input channels: 2 (stereo; mono input is duplicated by the runtime)
• Output: direct binaural stereo
• Output channel order: FL, FR
• Patch size: 200 samples
• Hidden size: 896
• Layers: 12
• Attention heads: 14
• Memory tokens: 32
• Mix-style conditioning dimensions: 10

The bundle defaults to RES6S with four steps, 10-second windows, 2-second overlap, and timestep-major traversal. Timestep-major inference carries model memory across every window at a shared solver timestep before advancing the solver, matching the long-audio training schedule.

Training Summary

This release contains 660,000 total training updates:

• 600,000 base-training updates with 10-, 18-, 26-, and 34-second sequences
• 60,000 long-context updates initialized from the base model's EMA weights, using 122-second sequences

Both stages used 10-second model windows with 2-second overlap, EMA with a decay of 0.999, and no adversarial loss. Mix-style conditioning used a 20% dropout probability, so the model supports both conditioned and unconditioned generation.

Mix-Style Conditioning

Mix-style conditioning is optional. Omitting a mix-style argument performs an unconditioned generation. The balanced preset is a conditioned neutral vector and is not the same as unconditioned generation.

For this direct-binaural model, the ten normalized conditioning dimensions are:

1. center_focus
2. center_lock
3. front_width
4. surround_amount
5. ambience_amount
6. placement_sharpness
7. spatial_contrast
8. chorus_expansion
9. lead_bloom
10. motion_amount

Quick Start

From a local checkout of stereo2spatial:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e .
python -m pip install -U "huggingface_hub[cli]"
hf download francislabounty/stereo2spatial-v2-binaural --local-dir checkpoints/stereo2spatial-v2-binaural
python infer.py --checkpoint checkpoints/stereo2spatial-v2-binaural --input-audio path\to\input.wav --output-audio path\to\output_binaural.wav --device cuda --show-progress

The command uses the sampler, step count, window, overlap, seed, and timestep-major traversal stored in config.json.

To use a conditioned preset:

python infer.py --checkpoint checkpoints/stereo2spatial-v2-binaural --input-audio path\to\input.wav --output-audio path\to\output_binaural.wav --device cuda --mix-style-preset balanced --show-progress

List the available presets with:

python infer.py --list-mix-style-presets

Bundle Contents

stereo2spatial-v2-binaural/
|-- config.json
|-- model.safetensors
|-- README.md
`-- LICENSE

Intended Use

This model is intended for:

• research and experimentation in stereo-to-binaural music generation
• local CUDA inference for headphone-oriented spatial audio
• prototyping immersive music and audio applications

It is not a replacement for professional mastering, broadcast authoring, or formal delivery-quality control.

Limitations

• The model produces direct binaural stereo for headphone playback. It does not produce a speaker layout or discrete surround channels.
• Training used stereo conditioning. Mono files are accepted by duplicating the channel, but mono was not part of this model's training distribution.
• The model is designed for music at 48 kHz. Other content and sample rates may produce weaker results even when the runtime resamples them. (Model was trained with a 15% chance for the input to be resampled to 44.1 kHz. So it should be able to handle 44.1 kHz reasonably well. But other sample rates were not part of the training distribution.)
• Mix-style values are normalized, corpus-relative controls rather than physical acoustic units.
• Results depend on the source and seed and may contain artifacts, unstable imaging, or undesirable balance changes. Manual listening and QC are required.

License

The model weights and bundle metadata are released under the Apache License 2.0. See LICENSE.

Acknowledgements

Compute for the stereo2spatial-v2 training run was provided by SERP, which supplied two NVIDIA RTX A6000 GPUs for approximately 20 days of training: about 10 days for Stage 1 and 10 days for Stage 2. Special thanks to SERP Apps, SERP AI, and Devin Schumacher for supporting the project.