relfx-ismir2026

RelFx ISMIR 2026

RelFx encodes the audio-effects transformation between two audio clips. Given a reference clip and a processed clip, it produces a normalized 128-dimensional projected representation that describes how their effects differ, even when the clips contain different musical content. The underlying 2048-dimensional fusion representation is also available for analysis.

This model accompanies the ISMIR 2026 paper "Beyond Dry References: Learning Relative Audio Effects Representations via Contrastive Distance Learning." The research source code is available at TMEGalaxyAudioEffect/relfx.

This release contains a MoisesDB-only Base Diff-Gate model. The checkpoint was retrained using the adjacent-pair sampling described in the paper. It achieves an average $L_d$ of 1.454, closely matching the 1.449 reported in Table 2. It is not the full-data or bidirectional checkpoint.

The checkpoint is publicly downloadable. Use, sharing, and adaptation remain subject to the terms in LICENSE.

Artifact identity

• File: model.safetensors
• Artifact format: relfx-safetensors-v1
• Training data: MoisesDB only
• Best epoch: 339
• Size: 405,072,104 bytes
• SHA-256: ea186f4b5a029ae3ba23a17e9d48dafb7c488e9f6636d0a407054ab5a986abc0
• Architecture: RelFx V6-fix, diff_gate, cross-attention stages [3, 5]
• Model variant: Base
• Final projected representation: 128 dimensions, L2-normalized
• Fusion representation: 2048 dimensions before projection

The release artifact contains 99 model tensors and JSON-encoded metadata in the Safetensors header. Optimizer and training-resume state are intentionally excluded.

Download

Install the Hugging Face CLI:

pip install -U huggingface_hub

Download the checkpoint directly; no account or authentication is required:

hf download TMEGalaxyAudioEffect/relfx-ismir2026 \
  model.safetensors \
  --local-dir weights/relfx-ismir2026

Use with the RelFx source release

git clone https://github.com/TMEGalaxyAudioEffect/relfx.git
cd relfx

python -m venv .venv
source .venv/bin/activate

pip install torch==2.7.1 torchaudio==2.7.1 \
  --index-url https://download.pytorch.org/whl/cu118
pip install -e .

export RELFX_CHECKPOINT_PATH=/absolute/path/to/model.safetensors
python scripts/verify_checkpoint.py "$RELFX_CHECKPOINT_PATH"

bash scripts/run_demo.sh \
  /path/to/reference.wav \
  /path/to/processed.wav \
  outputs/demo/embedding.npy

The two input files do not need to contain the same musical performance. RelFx resamples them to 44.1 kHz, converts them to stereo, and uses the first ten seconds.

The demo saves the final normalized 128-dimensional representation used by the paper. To export the raw 2048-dimensional fusion representation instead, invoke scripts/embed.py with --representation fusion.

Representation outputs

• model(...)["embedding"], model(...)["z"], and model.get_embedding(...) return the final normalized 128-dimensional representation used for contrastive learning, retrieval, and ITO.
• model(...)["fusion"] returns the 2048-dimensional e_fx before projection. This intermediate feeds the auxiliary parameter-regression head. Use model.get_fusion_representation(..., normalized=True) when an L2-normalized copy is needed directly.

Base and bidirectional variants

This checkpoint is the Base model from Eq. (4). Swapping its ordered inputs is not guaranteed to negate either representation. A manually negated Base output is not the paper's bidirectional model.

The source repository also implements the separately trained bidirectional architecture from Eqs. (6)-(7), including its symmetric gate, difference-only fusion, Tanh projection, and input-order swap training. No bidirectional checkpoint is distributed here. In that variant, fusion is strictly antisymmetric; the final normalized projection is only empirically approximately antisymmetric because its Linear layers contain learned biases.

Intended use

The model is intended for non-commercial research involving:

• audio-effects representation and retrieval;
• comparison of relative processing styles;
• analysis of effect transformations; and
• differentiable inference-time parameter matching.

Out-of-scope use

• Commercial products or services.
• Claims that the model recovers unique physical processor parameters.
• Source-wise effect disentanglement from a mixed recording.
• Processing audio without the necessary rights or consent.
• Redistribution or adaptation that does not comply with CC BY-NC-SA 4.0.

Training data

This released variant was trained on 2,585 stems from 240 MoisesDB songs (156.4 hours). It does not use the licensed internal collection described in the paper. Because MoisesDB does not provide the structural labels used by the other training sources, pairs are adjacent, non-overlapping 10-second clips sampled within the same stem.

Training audio and dataset metadata are not distributed.

Training and evaluation effects

The training chain contains EQ, distortion, multiband compression, gain, stereo imaging, limiting, delay, and reverberation. It has 72 continuous parameters plus activation switches.

The paper's inference-time parameter-matching evaluation uses a separate seven-effect, 47-parameter chain without reverb.

Stored validation metrics

The epoch-339 checkpoint records:

These are the validation metrics stored in the training checkpoint. Full parameter-matching results should be reported with the exact evaluation protocol and immutable model revision.

Parameter-matching check

Using the paper's normalized MUSDB18 Self-ref (wet_wet) protocol with 100 triplets per instrument, 200 iterations, and seven restarts, this checkpoint obtains the following median multi-resolution STFT losses:

Limitations

• RelFx estimates one global transformation between two clips.
• It does not disentangle independent source-wise effects inside a mixture.
• Evaluation uses a controlled differentiable DSP chain and does not cover every commercial plug-in or real-world preset.
• Relative embeddings are not unique or physically exact processor settings.

License

The checkpoint is released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0). Attribution, non-commercial use, and ShareAlike conditions apply; see LICENSE for the complete terms. Public availability does not replace these license obligations.

Citation

@inproceedings{liu2026beyond,
  title     = {Beyond Dry References: Learning Relative Audio Effects
               Representations via Contrastive Distance Learning},
  author    = {Liu, Xinlu and Lin, Huibin and Wei, Weixing and Yan, Zhenhai},
  booktitle = {Proceedings of the 27th International Society for Music
               Information Retrieval Conference (ISMIR)},
  year      = {2026}
}