who

Who

Label who said what in audio and video.

On-device speaker labeling: per-person turns with timestamps.

• SDKs, install and examples: https://desertant.com/docs/who/
• Website: https://desertant.com/models/who/

Label the speakers in a recording and see exactly when each person starts and stops talking. A 50-minute interview takes 3s on a MacBook Pro, entirely on the device.

Use the timestamps to label a transcript by speaker, jump to the next time a guest talks, or cut every time the speaker changes. Who is a speaker diarization model made for podcasts, interviews, calls and meetings, and handles recordings of any length.

Who numbers the speakers in the order they first speak, and when two people talk at once, Who marks both of them. Who processes each five-minute section in one pass and matches voices across sections. Who runs on the Neural Engine on Apple platforms and through LiteRT on Linux, Windows and Android, and there's also an ONNX build. The Core ML download is 81MB as float16 or 42MB with 8-bit weights, and both builds have the same error rate on AMI meetings, 12.6%.

Files

Each folder holds the same five model files for its runtime, plus its own copy of the four supporting files.

Inputs and outputs

• In: 16kHz mono audio of any length, from a few seconds up. The SDK converts other sample rates for you.
• Out: each speaker, numbered in the order they first speak, and when they start and stop talking. Times are in seconds, to the nearest 10ms. When two people talk at once, both are marked. Who labels up to four speakers in each five-minute section, and more across a whole recording.

Languages

Who works from the sound of each voice and has no language setting. Every score on this card was measured on English recordings.

Results

Measured through the SDK on the files in this repo, on whole recordings.

On VoxConverse, the error rate rises with the number of speakers in a recording.

Performance

Through the SDK.

On CPU alone, the 8-bit LiteRT build runs the same 544 minutes in 87s on the M5 Pro and 191s on a 24-core AMD EPYC server. These two times were measured outside the SDK.

Limitations

• More than four speakers in five minutes. Who splits a recording into five-minute sections and can label up to four speakers per section. Who matches voices across sections, so it can identify more than four speakers in a longer recording. On VoxConverse, the error rate is 8% with one or two speakers, versus 17% with five to eight and 25% with nine or more.
• Far-field recordings. Who misses more speech when the speakers are far away from the microphone. In group conversations recorded across the room, Who misses 19-21% of the speech, versus 8% in meetings where everyone wears a headset. Who does best on close-miked audio, such as podcasts, interviews and calls. On room recordings, expect stretches of speech with no speaker label.
• Overlapping speech. Who misses the most speech when two people talk at once. On AMI meetings, Who misses 8% of all speech, and about 65% of that is overlapping talk. Expect more misses in lively discussions with lots of interruptions.
• Short stretches of speech. Who mixes up speakers more often when many people each speak for only a few seconds. On Earnings-21 calls, with up to 20 speakers, Who assigns 16% of the speech to the wrong speaker. Expect more mix-ups on calls with lots of short remarks.
• Laughter, applause and jingles. Who can give a laugh, applause or a jingle its own speaker number. To hide these, set the SDK to drop any speaker with less than a chosen share of the recording's speech.
• Who labels speakers as 1, 2, 3 and so on, not by name, and doesn't transcribe. For a transcript labeled by speaker, pair Who with Voz.

Credits

Open models and tools this work builds on, with their licenses:

• pyannote-community/speaker-diarization-community-1, CC BY 4.0. The voice graph in this repo is its speaker embedding model, converted.

Dataset attributions are in THIRD_PARTY_NOTICES.md.

License

Desert Ant Labs Source-Available License. Free for most apps, and a commercial license is required at scale. Full terms are at the link. Licensing: [email protected].

See THIRD_PARTY_NOTICES.md.

Citation

@software{who_2026,
  title  = {Who: On-device speaker labeling: per-person turns with timestamps},
  author = {Desert Ant Labs},
  year   = {2026},
  url    = {https://huggingface.co/desert-ant-labs/who},
}

---

© 2026 Desert Ant Labs · https://desertant.com