Downloads · 30 days
21
22% of all-time downloads
jq/whisper-large-v2-salt-plus-xog-myx-kin-swa
whisper-large-v2-salt-plus-xog-myx-kin-swa is a automatic speech recognition model from jq. Use it when you need speech turned into text. It is set up for transformers.
Downloads · 30 days
21
22% of all-time downloads
All-time downloads
94
Public
Parameters
1.5B
111 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
Available languages:
| ISO 639-3 | Language |
|---|---|
| eng | English (Ugandan accent) |
| lug | Luganda |
| ach | Acholi |
| lgg | Lugbara |
| teo | Ateso |
| nyn | Runyankole |
| xog | Lusoga |
| myx | Lumasaba |
| swa | Swahili |
| kin | Kinyarwanda |
The model is used in a similar way to the base Whisper model. The model will attempt to auto-detect the language and provide a transcription. However, note that language detection is not always accurate and results may be improved by specifying it instead. The languages in this model are not supported by the base Whisper model, so the format is slightly different:
!git clone https://github.com/jqug/salt.git
import salt.constants
import transformers
import datasets
import torch
processor = transformers.WhisperProcessor.from_pretrained(
"jq/whisper-large-v2-salt-plus-xog-myx-kin-swa")
model = transformers.WhisperForConditionalGeneration.from_pretrained(
"jq/whisper-large-v2-salt-plus-xog-myx-kin-swa")
# Get some test audio
ds = datasets.load_dataset('Sunbird/salt', 'multispeaker-lug', split='test')
audio = ds[0]['audio']
sample_rate = ds[0]['sample_rate']
# Specify a language from: eng, lug, ach, teo, lgg, nyn, myx, xog, swa, kin.
lang = 'lug'
# Apply the model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
input_features = processor(
audio, sampling_rate=sample_rate, return_tensors="pt").input_features
input_features = input_features.to(device)
predicted_ids = model.to(device).generate(
input_features,
# Optionally set language=None here instead to auto-detect.
language=processor.tokenizer.decode(
salt.constants.SALT_LANGUAGE_TOKENS_WHISPER[lang]),
forced_decoder_ids=None)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription)
# Ekikoola kya kasooli kya kyenvu wabula langi yaakyo etera okuba eya kitaka wansi.