unispeech-sat-base-plus-sv-fp16

unispeech-sat-base-plus-sv-fp16

An fp16 copy of microsoft/unispeech-sat-base-plus-sv, Microsoft's UniSpeech-SAT speaker-verification encoder (X-Vector head, fine-tuned on VoxCeleb1). No changes beyond the dtype cast; reduced size and precision.

Conversion

Exactly:

import torch
from transformers import UniSpeechSatForXVector

encoder = UniSpeechSatForXVector.from_pretrained(
    "microsoft/unispeech-sat-base-plus-sv", dtype=torch.float32
)
encoder.half().save_pretrained(".")

(convert.py in this repo.)

Usage

from transformers import UniSpeechSatForXVector

encoder = UniSpeechSatForXVector.from_pretrained("Remsky/unispeech-sat-base-plus-sv-fp16")

Input is 16 kHz mono audio, as with the original. Load with dtype=torch.float32 if you want fp32 compute from the fp16 weights.

License and credits

All credit for the model goes to Microsoft; this repo is only a dtype conversion.

• Original model: microsoft/unispeech-sat-base-plus-sv, licensed under CC BY-SA 3.0 (the official license in the microsoft/UniSpeech repository). This derivative keeps the same license.
• Paper: UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training (Chen et al., 2021).

@article{chen2021unispeechsat,
  title={UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training},
  author={Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Chen, Zhengyang and Chen, Zhuo and Liu, Shujie and Wu, Jian and Qian, Yao and Wei, Furu and Li, Jinyu and Yu, Xiangzhan},
  journal={arXiv preprint arXiv:2110.05752},
  year={2021}
}