towards-speechaldi-finetuned-asr-whisper-medium

Whisper Medium — Fine-tuned for Arabic ASR on SADA

openai/whisper-medium fine-tuned on the full SADA22 dataset (~420 hours of Saudi Arabic speech) for Arabic automatic speech recognition (ASR).

Used as Baseline 2 in experiments on predicting the Arabic Level of Dialectness (ALDi) from speech: the transcript produced by this model is fed into a text-based ALDi classifier to obtain a dialect score.

---

Training details

---

Quick Start

1. Install dependencies

pip install torch "transformers>=4.27" torchaudio safetensors

2. Transcribe an audio file

import torch
import torchaudio
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model = WhisperForConditionalGeneration.from_pretrained("wageehkhad/whisper-medium-finetuned-sada-asr")
processor = WhisperProcessor.from_pretrained("wageehkhad/whisper-medium-finetuned-sada-asr")


def transcribe(audio_path: str, device: str = "cpu") -> str:
    """
    Transcribe an Arabic audio file.
    Accepts any format supported by torchaudio (WAV, FLAC, MP3, etc.).
    """
    wav, sr = torchaudio.load(audio_path)
    wav = torchaudio.functional.resample(wav, sr, 16_000).mean(0).numpy()

    inputs = processor(
        wav,
        sampling_rate=16_000,
        return_tensors="pt",
        return_attention_mask=True,
    ).to(device)
    model.to(device)

    with torch.no_grad():
        token_ids = model.generate(
            inputs.input_features,
            attention_mask=inputs.attention_mask,
            language="arabic",
            task="transcribe",
        )

    return processor.batch_decode(token_ids, skip_special_tokens=True)[0]

# Example
print(transcribe("example.wav"))

If you get the error:

ImportError: TorchCodec is required for load_with_torchcodec

make sure to run

pip install torchcodec

3. Using as part of the Baseline 2 ALDi pipeline

This model is the ASR component of a two-step pipeline:

Audio → [this model] → Arabic transcript → [AMR-KELEG/Sentence-ALDi] → ALDi score

from transformers import pipeline, AutoModelForSequenceClassification, AutoTokenizer
import torch

# Step 1: transcribe
asr = pipeline("automatic-speech-recognition",
               model="wageehkhad/whisper-medium-finetuned-sada-asr")
result = asr("speech.wav", generate_kwargs={"language": "arabic", "task": "transcribe"})
transcript = result["text"]

# Step 2: score dialect level
aldi_tok = AutoTokenizer.from_pretrained("AMR-KELEG/Sentence-ALDi")
aldi_mdl = AutoModelForSequenceClassification.from_pretrained("AMR-KELEG/Sentence-ALDi")
aldi_mdl.eval()

enc = aldi_tok(transcript, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    score = float(aldi_mdl(**enc).logits.squeeze())

print(f"Transcript : {transcript}")
print(f"ALDi score : {score:.3f}")  # 0.0 = MSA, 1.0 = heavy dialect

---

Limitations

• Fine-tuned on Saudi Arabic (SADA22). WER on other Arabic dialects or MSA broadcasts will be higher.
• Optimised for speech segments up to 30 seconds (Whisper's native window). Longer files are chunked automatically by the pipeline.

---

Citation

If you use this model, please cite the ALDi paper and the SADA dataset:

@inproceedings{khadawardi-etal-2026-towards,
    title = "Towards Speech {ALD}i: Predicting the {A}rabic Level of Dialectness Directly from Speech",
    author = "Khadawardi, Wageeh  and
      Keleg, Amr  and
      Magdy, Walid",
    booktitle = "Proceedings of The Fourth {A}rabic Natural Language Processing Conference",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.arabicnlp-main.28/",
    doi = "10.18653/2026.arabicnlp-main.28",
    pages = "409--417",
    ISBN = "979-8-89176-473-6",
    abstract = "Arabic Dialect Identification (ADI) aims at automatically assigning an utterance{---}written or spoken{---}to a single variety, be it Modern Standard Arabic (MSA) or a variety of Dialectal Arabic (DA). To complement ADI, the Arabic Level of Dialectness (ALDi) has been proposed to model the different dialect levels that exist between MSA and Highly Dialectal text. However, current ALDi estimation models operate on text inputs only. This paper presents the first step toward extending ALDi to the speech modality. First, we use the automatically estimated ALDi score of a spoken utterance{'}s transcript as the silver-standard target score. We then compare the performance of two approaches at automatically predicting these silver-standard ALDi scores from spoken utterances: (1) a cascaded approach where automatic speech recognition is used to generate automatic transcripts that are then fed to a text-based ALDi estimation model, and (2) a direct ALDi estimation from speech. We use SADA{---}a dataset of 435 hours of manually transcribed Arabic speech{---}as the fine-tuning source data, and evaluate the two approaches on three datasets that cover different varieties of Arabic. Our results indicate that estimating silver-standard ALDi scores directly from speech is on-par or slightly worse than the cascaded approach for in-domain data. However, the direct approach generalizes better, especially for dialects where the fine-tuned ASR backbone of the cascaded approach has relatively large word-error-rates. This paper paves the way towards predicting ALDi scores directly from speech inputs."
}

@inproceedings{keleg2023aldi,
  title     = {ALDi: Quantifying the Arabic Level of Dialectness of Text},
  author    = {Keleg, Amr and Goldwater, Sharon and Magdy, Walid},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2023},
  publisher = {Association for Computational Linguistics},
  address   = {Singapore},
  url       = {https://aclanthology.org/2023.emnlp-main.655}
}