tadabur-whisper-medium

🕌 Tadabur — Quran Speech Recognition

Fine-tuned Whisper Medium on the Tadabur dataset for Quran ASR, Surah/Ayah identification, and reciter recognition.

CS465 Machine Learning Project — Spring 2026

v2 (current). Trained on the full corpus — 1,105 hours, 339,473 clips — with reciter- and surah-disjoint evaluation. v1 was trained on a single 9,432-sample shard and evaluated on a split from that same shard; it remains available at revision v1-single-shard.

---

What This Model Does

Given a Quran audio recitation, the pipeline returns:

1. Arabic transcription — 4.46% WER on surahs the model never trained on
2. Surah & Ayah identification — fuzzy matched against all 6,236 ayahs
3. Reciter name — from 335 supported reciters, 91.84% top-1

---

Performance

WER and CER use undiacritized normalization (tashkeel stripped, alef/ya/ta-marbuta unified, punctuation dropped). The training target was text_ar_simple. Diacritized WER would be considerably higher.

ASR

The first two rows describe real accuracy. The Quran is a closed corpus of 6,236 ayahs, so a random split puts the same text on both sides and the decoder's language prior does much of the work — which is what produces 0.33% and 0.67%. On text the model has never transcribed, error settles near 4.5%.

Unseen voices cost far less than unseen text (0.67% vs 4.46%). Whisper's encoder was pretrained on ~680k hours of varied speech, so ten unfamiliar reciters barely register; corpus familiarity in the decoder is the larger effect.

Comparison against the base model

Vanilla openai/whisper-medium evaluated on the same splits (1,500 clips per split, identical normalization):

Note that vanilla does worse on held-out reciters (18.09%) than on held-out surahs (11.40%) — the reverse of the fine-tuned ordering. The held-out reciters are therefore genuinely harder audio, not easier. That makes v2's 0.67% on that split direct evidence for the language-prior reading above: the advantage comes from knowing the text, not from favourable acoustics.

Vanilla's un-normalized WER is 31.99%, against 11.78% normalized — it emits different orthography and diacritics, so normalized figures are the fair comparison for both models.

Comparison against Tadabur-Whisper-Small

Tadabur-Whisper-Small (Alherran, released with the dataset) evaluated on the same splits:

† Trained on the full Tadabur corpus. These splits are held out from this model's training, not from his — so surahs 12/32/48/67/86 and the ten held-out reciters are likely in-distribution for Tadabur-Whisper-Small. Its figures are therefore not held-out results. * Held out from this model's training only.

Read this table carefully rather than as a ranking. On test_surah the small model scores better (2.15% vs 4.46%) — but that is a seen-text number against an unseen-text number, which is not a like-for-like comparison. The only row where both models face the same condition is test_seen, where both have seen the data during training: 0.33% versus 2.02%.

One detail resists a clean explanation: if Tadabur-Whisper-Small trained on everything, its fall to 12.43% on test_reciter is steep. That mirrors the vanilla baseline's pattern (18.09% there against 11.78% on test_seen), so those reciters are genuinely harder audio — but it leaves open whether they were in its training data. Its model card does not state a split, so this is unresolved.

Settling it properly would need either Alherran's split definition or an evaluation set neither model trained on.

v1 reported 6.26% WER against a 41.10% vanilla baseline and 47.06% for Tadabur-Whisper-Small (Alherran's model, released with the dataset). Those came from v1's 500-clip same-distribution split and are not comparable to the table above.

Is the model memorizing the Quran?

A fair worry for a fixed corpus: a decoder could emit canonical ayahs from weak acoustic evidence and score well without listening. Tested by truncating audio to 50% and measuring how many reference words still appear.

Half the audio costs half the words, by nearly the same margin either way. A model reciting from memory would have stayed near 99% on familiar text. No memorization signal — the seen/unseen gap is a language prior, not retrieval.

Reciter Classifier

The held-out-surah column is the one to trust. A random split lets the same reciter's reading of the same surah fall on both sides, leaking recording-session characteristics — microphone, room, mastering. v1's 98.47% was measured under that weaker protocol.

Accuracy depends heavily on how much audio a reciter contributes:

For reciters in the sparse tail, prefer top-5 or a confidence threshold.

---

Files in This Repository

---

Quick Start

Install

pip install transformers torch librosa rapidfuzz huggingface_hub

Transcription only

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa, torch

MODEL = "rakansuliman/tadabur-whisper-medium"
processor = WhisperProcessor.from_pretrained(MODEL)
model = WhisperForConditionalGeneration.from_pretrained(MODEL)
model.eval()

audio, _ = librosa.load("recitation.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt").input_features

with torch.no_grad():
    ids = model.generate(
        inputs,
        language="arabic",
        task="transcribe",
        max_new_tokens=225,
        suppress_tokens=[],
        forced_decoder_ids=None,
    )

text = processor.batch_decode(ids, skip_special_tokens=True)[0]
print(text)

Audio must be 16 kHz mono. Whisper's window is a fixed 30 seconds — segment longer recordings before transcribing.

Reciter identification

v2 changes how embeddings are pooled. The classifier was retrained with pooling masked to frames that actually contain audio. Whisper pads every clip to 30 s, and with a median clip near 13 s the old unmasked .mean(dim=1) averaged mostly silence. Code written for v1 will still run against v2 weights but will lose accuracy.

from huggingface_hub import hf_hub_download
import torch, torch.nn as nn, json

MODEL = "rakansuliman/tadabur-whisper-medium"

hf_hub_download(MODEL, "reciter_classifier.pt",  local_dir="./")
hf_hub_download(MODEL, "reciter_idx_to_id.json", local_dir="./")
hf_hub_download(MODEL, "sheikh_dict.json",        local_dir="./")

class ReciterClassifier(nn.Module):
    def __init__(self, hidden_dim, num_classes):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(hidden_dim, 512), nn.BatchNorm1d(512), nn.ReLU(), nn.Dropout(0.3),
            nn.Linear(512, 256), nn.BatchNorm1d(256), nn.ReLU(), nn.Dropout(0.2),
            nn.Linear(256, num_classes),
        )
    def forward(self, x): return self.net(x)

with open("reciter_idx_to_id.json") as f:
    idx_to_id = {int(k): int(v) for k, v in json.load(f).items()}
with open("sheikh_dict.json", encoding="utf-8-sig") as f:
    sheikh = {int(v): k for k, v in json.load(f).items()}   # name→id, inverted

clf = ReciterClassifier(1024, len(idx_to_id))
clf.load_state_dict(torch.load("reciter_classifier.pt", map_location="cpu"))
clf.eval()

with torch.no_grad():
    hidden = model.model.encoder(inputs).last_hidden_state    # (1, 1500, 1024)
    # mask to real audio: each encoder frame covers 20 ms
    n_frames  = min(1500, max(1, int(len(audio) / 16000 / 0.02)))
    embedding = hidden[:, :n_frames].mean(dim=1).float()
    logits     = clf(embedding)
    pred_idx   = logits.argmax(dim=1).item()
    confidence = torch.softmax(logits, dim=1).max().item()

reciter_id   = idx_to_id[pred_idx]
reciter_name = sheikh.get(reciter_id, f"ID {reciter_id}")
print(f"Reciter: {reciter_name} ({confidence*100:.1f}%)")

Surah & Ayah identification

from rapidfuzz import process, fuzz

hf_hub_download(MODEL, "quran_simple.json", local_dir="./")
hf_hub_download(MODEL, "surah_dict.json",   local_dir="./")

with open("quran_simple.json", encoding="utf-8") as f:
    quran = json.load(f)          # all 6,236 ayahs
with open("surah_dict.json", encoding="utf-8-sig") as f:
    surah_names = json.load(f)

match, score, _ = process.extractOne(text, quran, scorer=fuzz.token_sort_ratio)
print(f"Matched ayah: {match}  (confidence {score:.0f}%)")

Match quality follows transcription quality, so expect it to be weakest exactly where WER is highest — on unfamiliar recitations.

---

Architecture

Audio Input (mic / file / video)
    ↓
Whisper Encoder  ←─ runs once, shared
    ├── Whisper Decoder  →  Arabic text
    └── MLP Classifier   →  Reciter name
    ↓
RapidFuzz matching against 6,236 ayahs
    ↓
Surah name + Ayah number + confidence

Reciter Classifier

Linear(1024→512) → BatchNorm → ReLU → Dropout(0.3)
    → Linear(512→256) → BatchNorm → ReLU → Dropout(0.2)
    → Linear(256→335)

---

Training Details

ASR Fine-tuning

• Base model: openai/whisper-medium — started fresh, not continued from v1
• Dataset: 339,473 clips / 1,105 hours (full Tadabur corpus)
• Hardware: NVIDIA RTX 4090 (24 GB), ~13 hours
• Epochs: 2 — 21,218 steps
• Batch size: 16 × 2 gradient accumulation = 32 effective
• Learning rate: 1e-5 cosine, 800 warmup steps
• Precision: bf16 (fp32's exponent range; avoids the loss spikes and NaNs fp16 Whisper fine-tuning is prone to)
• Gradient checkpointing enabled
• Published checkpoint: step 21,218

Data preparation

From the 938 GB source: decoded, resampled to 16 kHz mono, re-encoded FLAC; dropped clips over 30 s (Whisper's window) and under 0.5 s, and transcripts over 440 tokens (decoder limit 448). 361,583 of ~385k clips survive — 1,105 h of the corpus's 1,400 h, since the dropped clips were the longest ones.

Evaluation splits

Held-out reciters were drawn from the mid-frequency band — enough audio for a meaningful test set without removing a major contributor from training.

Reciter Classifier

• Encoder frozen; embeddings extracted once with masked mean pooling
• Capped at 1,000 clips per reciter for class balance — 168,662 embeddings
• MLP trained 20 epochs, AdamW 1e-3, cosine annealing, batch 256
• The 335-reciter mapping is unchanged from v1

---

Supported Reciters

See supported_reciters.txt for all 335, including: عبد الباسط عبد الصمد، محمد صديق المنشاوي، ياسر الدوسري، سعود الشريم، ماهر المعيقلي، عبدالرحمن السديس، and 329 more.

---

Limitations

• Audio over 30 seconds is unsupported — segment first
• Undiacritized output only; Uthmani script and tashkeel were not trained
• ~4.5% WER on unfamiliar text, against 0.33% in-distribution. Benchmarks built on random splits of this corpus will overstate real accuracy
• Reciter ID is unreliable in the sparse tail — 64.9% mean top-1 below 100 training clips. Six reciters (IDs 25, 89, 252, 280, 294, 381) score 0% top-1 despite adequate data; likely duplicate identities in the source, unconfirmed
• 31 of 335 reciters could not be evaluated under the held-out-surah protocol, having contributed fewer than 3 distinct surahs
• Reciter classifier covers 335 of the 671 reciters in the dataset
• Surah/Ayah matching accuracy depends on transcription quality
• Optimized for standard Hafs recitation; other qira'at untested
• Checkpoint 21,218 was published over the trainer-selected step 18,000, which scored better on held-out reciters but worse on held-out surahs (5.08% vs 4.46%)

---

Versions

---

Citation

@misc{suliman2026tadabur,
  author = {Suliman, Rakan and Mamdoh, Abdulrahman and Aldosari, Hussam and Khalid, Mohammed},
  title  = {Tadabur: Quran ASR with Surah/Ayah Identification and Reciter Recognition},
  year   = {2026},
  url    = {https://huggingface.co/rakansuliman/tadabur-whisper-medium}
}

Please also cite the dataset this model was trained on:

@misc{alherran2026tadabur,
  author        = {Alherran, Faisal},
  title         = {Tadabur: A Large-Scale Quran Audio Dataset},
  year          = {2026},
  eprint        = {2604.18932},
  archivePrefix = {arXiv},
  primaryClass  = {cs.SD},
  doi           = {10.48550/arXiv.2604.18932},
  url           = {https://arxiv.org/abs/2604.18932}
}

---

License

CC BY-NC 4.0 — Research and educational use only. Please engage with Quran content respectfully. 🤲