Downloads · 30 days
88
5% of all-time downloads
IbrahimDayax/xls-r-1b-somali-stt-asr
xls-r-1b-somali-stt-asr is a automatic speech recognition model from IbrahimDayax. Use it when you need speech turned into text. The card lists the license as cc-by-nc-4.0.
<p align="center" <bWER 6.83% · CER 1.42%</b<br <subnormalized (lowercased, punctuation-stripped) text · 781-row verified-clean, held-out test split</sub<br <subthe lowest error rate of every pub…
Downloads · 30 days
88
5% of all-time downloads
All-time downloads
1.8K
Public
Parameters
963M
3.9 GB on disk
Likes
4
Public
Click a slice to open those files.
.safetensors3.9 GB · 100%
From the Hugging Face model README
A fine-tune of facebook/wav2vec2-xls-r-1b for Somali speech-to-text. This is an ASR model only — it does not do text-to-speech, and no TTS claim is made anywhere in this card.
2026-07-18: benchmarked against every other public Somali ASR model on the Hugging Face Hub we could locate and load, on the same held-out test set. See § Comparison to other public Somali ASR models. Training data/config is unchanged from the original release — this update is evaluation-only.
2026-07-19: added two more models to that comparison —
openai/whisper-large-v3(a general multilingual foundation model, not Somali-finetuned; Somali is one of its officially supported languages, but it still underperforms every Somali-specific model here) andQwen/Qwen3-ASR-1.7B-hf(Somali is not one of its supported languages — included as an out-of-distribution check, not a fair comparison). Neither addition changes the ranking or the margin over the field; see below.
| Architecture | Wav2Vec2 XLS-R, CTC head |
| Parameters | 0.96B |
| Base checkpoint | facebook/wav2vec2-xls-r-1b (self-supervised encoder, no CTC head) |
| Tokenizer / vocabulary | borrowed from Mustafaa4a/ASR-Somali, 33 character-level tokens |
| Language | Somali (so) |
| Audio input | 16 kHz mono |
| Max clip length (training) | 30s |
| License | CC-BY-NC-4.0 |
Full fine-tune (not LoRA/adapter), single stage, on an A100/H100-class GPU:
| Epochs | 15 |
| Learning rate | 3e-5 |
| Batch size | 32 (native, no gradient accumulation) |
| Warmup ratio | 0.1 |
| Precision | bf16 |
| Feature extractor | frozen |
| Training data | IbrahimDayax/somali-asr-synthetic-youtube (full) + IbrahimDayax/somali-combined-asr-stt-dataset (train split only) |
The tokenizer was deliberately borrowed from Mustafaa4a/ASR-Somali rather than
built fresh from the training corpus — see § Known limitations for the
consequence (an unused en-dash token, silently dropped ASCII hyphens).
We searched the Hugging Face Hub for every model tagged or named for Somali
ASR, excluded repos with no usable model weights (ooloteam10/wav2vec2-somali,
ooloteam10/somali_wav2vec2, Mustafaa4a/ASR-Somali-v2 — all empty), and
benchmarked the rest on the same 781-row verified-clean test split from
§ Evaluation, using the same text normalization (lowercased, punctuation
stripped) for every model, so the numbers below are directly comparable.
| Model | Architecture | WER | CER |
|---|---|---|---|
| This model | Wav2Vec2 XLS-R 1B, CTC | 6.83% | 1.42% |
Mustafaa4a/ASR-Somali | Wav2Vec2, CTC | 35.9% | 9.7% |
ooloteam/wav2vec2-somali | Wav2Vec2, CTC | 38.9% | 9.8% |
laalays/mms1b-finetuned-somali-2 | MMS-1B + Somali adapter, CTC | 41.8% | 10.1% |
skydheere/wav2vec2-large-mms-1b-somalia | MMS-1B + Somali adapter, CTC | 46.2% | 14.6% |
khederwaaOne/whisper-small-somali-01 | Whisper-small, seq2seq | 67.0% | 23.9% |
steja/whisper-small-somali | Whisper-small, seq2seq | 73.7% | 23.3% |
tacab/tacab_asr_somali | Wav2Vec2, CTC | 77.4% | 28.7% |
openai/whisper-large-v3 | Whisper-large-v3, seq2seq — general multilingual, not Somali-finetuned | 92.7% | 31.0% |
ayaazlion/asr-somali-may | Wav2Vec2 XLS-R 300M, CTC | 95.2% | 34.7% |
laalays/mms1b-finetuned-somali | MMS-1B, CTC (no adapter) | 100.0% | 83.6% |
Qwen/Qwen3-ASR-1.7B-hf | Qwen3-ASR (Qwen3-Omni backbone) — Somali not a supported language | 115.5% | 85.2% |
~5.3x lower WER than the next-best model we could benchmark (Mustafaa4a/ASR-Somali,
35.9%) — not a marginal edge.
Notes on the table, in the interest of not overstating this:
somali-combined-asr-stt-dataset (the test
split scored here was held out and never seen — see § Training
configuration and § Evaluation) — so it has seen the same source
distribution the test split is drawn from, an advantage none of the other
models in this table have, since none of them were trained on any part of
this dataset. Worth weighing when reading the margin below.steja/whisper-large-somali
could not be benchmarked — it failed to load with the tokenizers
library version in our environment (argument 'special_tokens': Expected Union[Tuple[str, int], Tuple[int, str], dict]), an incompatibility with an
older repo format, not a measurement of model quality. It is the one
reasonably prominent public Somali ASR checkpoint missing from this table.laalays/mms1b-finetuned-somali's 100% WER was verified by hand — it
produces incoherent, repetitive character strings (e.g. qayb → qisibssn)
under standard loading, not a scoring artifact.openai/whisper-large-v3
is the fairest general-purpose reference point in this table: it wasn't
fine-tuned for Somali at all, so its 92.7% WER is a read on how far a strong
multilingual foundation model — at ~3.2x this model's parameter count —
gets on Somali zero-shot. Not far, and clearly behind every Somali-specific
model tested except the two broken/unsupported ones below it.Qwen/Qwen3-ASR-1.7B-hf's
115.5% WER (WER can exceed 100% when the hypothesis has far more insertions
than the reference has words) is expected, not a quality signal: Somali is
not among its 30 officially supported languages, and in our runs it
auto-detected Arabic or Malay and transliterated the audio accordingly
rather than transcribing Somali. Included as an out-of-distribution check
only.Somali-tts/somali_tts_model, saleolow/somali-mms-tts
— that this card makes no claim about).IbrahimDayax/somali-combined-asr-stt-dataset (current, primary)This dataset combines three Somali speech sources (jelle8000, soomali_asr,
somali_tts), deduplicated by transcript text. Its train split was part of
this model's training data (see § Training configuration) — but its test
split, scored below, was held out and never seen during training, so this
remains a genuine held-out evaluation and the basis for every number in
§ Comparison above. The 811-row test split was verified before scoring: each
row must have non-empty text, audio that actually decodes, audio ≥0.2s, and a
transcript free of two formatting artifacts found in a minority of rows —
somali_tts rows with words joined by _ instead of a space (e.g.
Nadaafadda_gacmaha_waxay_...), andsoomali_asr rows with an embedded |-separated grammatical tag that was
never spoken (e.g. Waa neef|m., hal|f — m/f/pl/prn markers).30 of 811 rows failed verification (16 tag-annotated, 14 underscore-joined) and were excluded. All 781 surviving rows were scored — no subsampling.
WER 6.83% · CER 1.42% (normalized text: lowercased, punctuation stripped — the fair comparison, since this model's 33-token vocabulary has no capitalization or punctuation tokens to begin with).
Per-utterance: 635/781 (81.3%) transcribed perfectly (WER=0); 34/781 (4.4%) at WER>50%. The per-utterance mean (7.3%) sits above the median (0%) — most of the test set is exact, and the mean is pulled up by a long tail of very short utterances (median reference length: 3 words) where a single error is a large percentage.
This model's tokenizer/vocabulary was bootstrapped from Mustafaa4a/ASR-Somali
— so beyond its place in the broader comparison above, it's worth calling out
as the direct "did fine-tuning actually help?" baseline. Both were scored on
the same 781 clean rows.
| Model | WER | CER |
|---|---|---|
facebook/wav2vec2-xls-r-1b + randomly-initialized CTC head (noise floor, not a real system) | 351.4% | 263.4% |
Mustafaa4a/ASR-Somali (prior Somali ASR) | 35.9% | 9.7% |
| This model | 6.83% | 1.42% |
81% relative WER reduction (35.9% → 6.83%) over the prior Somali ASR this model's vocabulary came from. The random-head row is included only as a sanity floor — it measures an untrained decoder, not the pretrained encoder, and isn't a competing system.
The evaluation set (IbrahimDayax/somali-combined-asr-stt-dataset) is
gold-standard: its somali_tts rows are gold-standard by construction (the
text was the input to speech synthesis, not derived from the audio), and its
jelle8000 and soomali_asr rows are human-annotated. The WER/CER above
therefore measure correctness against real transcripts, not agreement with
another ASR system's bootstrapped output.
This model's training data (IbrahimDayax/somali-asr-synthetic-youtube), by
contrast, is silver-standard — produced by ASR bootstrapping from YouTube, not
human annotators. That's a property of the training data, unrelated to how
this evaluation was scored.
2024; numerals must be
spelled out. Training transcriptions contained no digits.Mustafaa4a/ASR-Somali,
not built from this model's own training corpus) contains an en-dash (–,
U+2013) that never appears in the training data, while the training data's
ASCII hyphens (-, U+002D) were silently stripped from the labels — so
hyphenation is dropped and the en-dash token is never produced.af-Carabi →
afcarabi; cabbid → nabbid). Beam-search decoding with a 5-gram KenLM
model (pyctcdecode, trained on this model's own ~250k-word training-split
transcripts) was measured, not assumed: it moved WER from 6.83% → 7.28%
(worse), and after a 16-point alpha/beta sweep tuned on a held-out validation
split, the best configuration only reached 8.20% vs. a greedy baseline of
8.31% on that same split — a ~1% relative gain, not the 15–30% sometimes
quoted for this technique. At this model's current error rate there is little
left for a thin, same-domain n-gram model to fix; a substantially larger and
more general external Somali text corpus (news, Wikipedia, books) is the more
promising lever than tuning this one further.steja/whisper-large-somali is not in the § Comparison table — it failed
to load in our environment (tokenizer format incompatibility), not because it
was tested and excluded. Treat the ranking as "best of what loaded," not
exhaustive.import librosa
import torch
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
model_id = "IbrahimDayax/xls-r-1b-somali-stt-asr"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id).eval()
speech, _ = librosa.load("somali_audio.wav", sr=16_000) # 16 kHz mono required
inputs = processor(speech, sampling_rate=16_000, return_tensors="pt")
with torch.no_grad():
logits = model(inputs.input_values).logits
transcription = processor.batch_decode(torch.argmax(logits, dim=-1))[0]
print(transcription)