Speaker-Diarization-Models

<!-- README for the Hugging Face repo `42ailab/Speaker-Diarization-Models`, uploaded with the diarization model package. Editing this file changes the repo landing page. Multi-source models: pyannote segmentation (MIT) + WeSpeaker embeddings (Apache-2.0). -->

Speaker Diarization Models · Who Spoke When (Fully Local)

English | 简体中文 · Also on ModelScope · 42ailab.com

Give it a multi-speaker conversation recording and this model marks who said each segment (Speaker 1, Speaker 2, …) — entirely on your own computer. No cloud, free, private.

The models were built by two open-source teams: pyannote (speech segmentation, MIT) and WeSpeaker (speaker embeddings, Apache-2.0); the ONNX deployment packaging follows avencera/speakrs (Apache-2.0). This repo is not a new model — we converted these open models into deployment builds that run offline on your computer and verified their quality. It is an enhancement model and does not transcribe: transcription is still done by Qwen3-ASR, and this model adds the "who is speaking" information on top.

1. The problem it solves

Speech-to-text tells you "what was said", but in a multi-speaker conversation it often cannot tell you who said this line. Many scenarios depend precisely on that distinction:

• Meeting minutes: organize notes by speaker so each party's position is visible at a glance;
• Interviews and podcasts: separate host from guest, making transcripts and quotations easier to prepare;
• Customer service and QA: distinguish agent from customer and analyze each separately.

Doing speaker diarization used to mean uploading the recording to a cloud service — with both privacy concerns and possible costs. This model keeps that step entirely local.

2. How it works

It chains three steps into a "who spoke when" pipeline:

• Segmentation (pyannote): first find which stretches of audio contain speech, and where the speaker changes;
• Speaker embeddings (WeSpeaker): extract a "voiceprint" vector representing the voice characteristics of each short segment;
• Clustering: group segments with similar voiceprints as the same person, and finally attach a speaker label to each segment.

What we did: converted the upstream open weights into locally runnable deployment builds, verified that quality did not degrade in conversion, and made them run offline across macOS / Windows / Linux; we also selected the right acceleration path for Apple silicon (CoreML), NVIDIA GPUs (CUDA) and plain CPU, so machines without a discrete GPU can use it too.

3. How well it performs

For the complete diarization-quality benchmarks (DER, lower is better), refer to upstream public evaluations — per speakrs benchmarks, this pipeline (CoreML) achieves about 7.1% DER on the VoxConverse Dev subset, essentially on par with pyannote community-1 (about 7.2%). Here we list only our own verification:

Note: "correctness", "speed" and "memory" above are our own measurements on an Apple M3 Max; "7.1% DER (VoxConverse Dev)" is a public figure from upstream speakrs (a third-party CoreML port's evaluation — neither pyannote official nor measured by us).

4. Limitations and what's next

• Requires a transcription model: this model only answers "who is speaking" and does not transcribe; use it together with Qwen3-ASR.
• Plain-CPU speed: extremely fast with Apple / NVIDIA acceleration; about 1.9× real time on plain CPU — usable, but clearly slower than the accelerated tiers, so process long recordings on an accelerated machine where possible.
• Overlapping speech: where two people talk at once, speaker attribution may be imprecise (a shared difficulty of speaker diarization).
• Next: keep improving plain-CPU speed and handling of overlapping segments.

5. How to download and use it

This model is packaged for 42model, which is the recommended way to get it:

Desktop app

1. Open Model Library → Transcription and download Qwen3-ASR Enhanced (speaker diarization is built in);
2. When transcribing, turn on Speaker diarization in the parameter settings, and results will be segmented by speaker.

Files and license

Per-file sha256 values are in the bundled manifest.json and on the Files page, and can be verified independently. CoreML acceleration on Apple silicon is compiled automatically on first local run — no extra files needed.

License: this repo contains local deployment builds of open models from several sources — speech segmentation is pyannote/segmentation-3.0 (© pyannote.audio, MIT; the upstream HF page requires accepting terms for access, but the license is MIT and redistribution is unrestricted — the ONNX here comes from the same weights under the same license); speaker embeddings are WeSpeaker voxceleb ResNet34 (© the WeNet community, Apache-2.0); the ONNX packaging and pipeline follow avencera/speakrs (Apache-2.0). Each file is governed by its original license; by using them you agree to the respective upstream license terms.

Citation

For the models themselves, please cite upstream pyannote and WeSpeaker:

@inproceedings{Plaquet23,
  author    = {Alexis Plaquet and Herv{\'e} Bredin},
  title     = {{Powerset multi-class cross entropy loss for neural speaker diarization}},
  booktitle = {Proc. INTERSPEECH 2023},
  year      = {2023},
}
@inproceedings{Bredin23,
  author    = {Herv{\'e} Bredin},
  title     = {{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},
  booktitle = {Proc. INTERSPEECH 2023},
  year      = {2023},
}
@inproceedings{wang2023wespeaker,
  author    = {Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},
  title     = {{Wespeaker: A research and production oriented speaker embedding learning toolkit}},
  booktitle = {ICASSP 2023},
  year      = {2023},
}

If this repo's local deployment packaging was useful to you, you may additionally cite:

@misc{yang2026diarizationlocal,
  title        = {Speaker Diarization Models: A Local Build of pyannote + WeSpeaker for On-Device Diarization},
  author       = {Yang, Zhiping},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/42ailab/Speaker-Diarization-Models}},
  organization = {42ailab},
  note         = {Local deployment packaging (ONNX + cross-platform acceleration); the models themselves are pyannote/segmentation-3.0 (MIT) and WeSpeaker voxceleb ResNet34 (Apache-2.0). Contact: [email protected]}
}

Contact us: [email protected]

About us

42ailab — an AI research lab exploring the boundaries of intelligence. Grounded in cognitive science, we work toward a deep integration of AI and human intelligence — to truly understand and augment intelligence, carbon-based and silicon-based alike.

42model — a high-performance local inference engine from 42ailab that runs translation, transcription, recognition, chat and coding on your own machine, free and private; with optional cloud compute for fine-tuning your own models and bringing them back to run locally.