SENSE

SENSE multilingual semantic speech embeddings

This repository provides a pretrained SENSE model for extracting multilingual semantic speech embeddings.

The model was trained on 90 Common Voice languages following the SENSE framework described in:

SENSE models: an open source solution for multilingual and multimodal semantic-based tasks
https://arxiv.org/abs/2509.12093

Pipeline description

SENSE produces utterance-level semantic speech embeddings from 16 kHz audio.

Compute SENSE embeddings

import torch
from transformers import AutoFeatureExtractor, AutoModel
import soundfile as sf

processor = AutoFeatureExtractor.from_pretrained("LIA-AvignonUniversity/SENSE")
model = AutoModel.from_pretrained("LIA-AvignonUniversity/SENSE", trust_remote_code=True).eval()
audio_path = "audio.wav"
waveform, sample_rate = sf.read(audio_path)
inputs = processor(waveform, sampling_rate=16000, return_tensors="pt")
with torch.inference_mode():
	embedding = model(**inputs).embeddings

Training

The model was trained with SpeechBrain on multilingual Common Voice data.

The complete training recipe is available here:

https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice/SENSE

Referencing SENSE

If you use this model, please cite:

@article{mdhaffar2025sense,
  title={SENSE models: an open source solution for multilingual and multimodal semantic-based tasks},
  author={Mdhaffar, Salima and Elleuch, Haroun and Chellaf, Chaimae and Nguyen, Ha and Est{\`e}ve, Yannick},
  journal={ASRU},
  year={2025}
}

About SpeechBrain

• Website: https://speechbrain.github.io/
• Code: https://github.com/speechbrain/speechbrain/
• Hugging Face: https://huggingface.co/speechbrain/