magda-sample-tagger

MAGDA Sample Tagger

ONNX exports of LAION's CLAP HTSAT-unfused model plus the RoBERTa tokenizer, packaged for the MAGDA DAW's sample library (issue #768).

What's in this repo

• clap_audio.onnx — audio encoder. Takes a mono 48 kHz waveform, produces a 512-d normalised embedding suitable for cosine similarity search.
• clap_text.onnx — text encoder. Takes RoBERTa token ids + attention mask, produces a 512-d normalised embedding in the same space as the audio encoder so a text query can rank audio files by similarity.
• tokenizer.json — the RoBERTa BPE tokenizer that pairs with the text encoder. MAGDA's C++ tokenizer reads this file directly.

How MAGDA uses these

MAGDA's media database (a SQLite catalogue of audio samples) uses these encoders to:

• Compute an embedding per indexed sample at index time, stored in the media_embedding table.
• Encode the user's free-text search query at query time and rank samples by cosine similarity to the query embedding.

Without these models MAGDA falls back to filename / tag full-text search — still useful, just no semantic similarity.

Export procedure

ONNX exports are generated from laion/clap-htsat-unfused via the export script in MAGDA's prototype:

prototypes/media_db/src/media_db/embeddings/onnx_export.py

Notes:

• Run on CPU (MPS does not support float64 used by the audio encoder's mel filterbank).
• Requires transformers >= 5.x. The audio-feature accessor was renamed from audios= to audio= between 4.x and 5.x; passing the old kwarg silently returns wrong shapes.
• tokenizer.json is the unmodified file from the upstream HF repo, fetched via AutoTokenizer.from_pretrained(...).save_pretrained(...).

License

BSD-3-Clause — same as the upstream LAION CLAP weights. See the upstream repo for the original notice and attribution.