Downloads · 30 days
19
1% of all-time downloads
multilingual-distilwhisper-28k
naver/multilingual-distilwhisper-28k
multilingual-distilwhisper-28k is a automatic speech recognition model from naver. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as mit.
Multilingual Distilwhisper allows for better ASR performance in target languages by adding lightweight CLSR modules on top of whisper-small. These modules are trained on a mix of cross-entropy (ASR) and knowledge dist…
All-time downloads
1.3K
Public
Parameters
242M
4.7 GB on disk
Likes
15
Public
Repo makeup
Click a slice to open those files.
.bin2.3 GB · 71%
At a glance
- Task
- Automatic Speech Recognition
- Library
- transformers
- License
- mit
- Model type
- distilwhisper
- Access
- Public
- Created
- Nov 30, 2023
- Updated
- Aug 26, 2024
- SHA
- b3fc9572
Model card
From the Hugging Face model README
About
Multilingual Distilwhisper allows for better ASR performance in target languages by adding lightweight CLSR modules on top of whisper-small. These modules are trained on a mix of cross-entropy (ASR) and knowledge distillation losses, where whisper-large-v2 is used as teacher. More details in the ICASSP 2024 paper: arxiv.org/abs/2311.01070
Inference
Code for training and inference at: https://github.com/naver/multilingual-distilwhisper
Citation
@inproceedings{ferraz2024distilwhisper,
title={Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Experts},
author={Ferraz, Thomas Palmeira and Boito, Marcely Zanon and Brun, Caroline and Nikoulina, Vassilina},
booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
year={2024},
organization={IEEE}
}
- Task
- Automatic Speech Recognition
- Library
- transformers
- Type
- distilwhisper
- License
- mit
- Languages
- ca, ta, th
- Created
- Nov 30, 2023
- Updated
- Aug 26, 2024