WhisperLv3-EP-X

WhisperLv3-EP-X: Whisper large-v3 fine-tuned for European Portuguese (WhisperX) — pré-Acordo

Whisper large-v3 fine-tuned on 425 h of European Portuguese (EP-425), packaged for WhisperX (CTranslate2, VAD-based segmentation). This is the best EP model in the paper and the recommended one for long-form audio.

From the paper CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese (INESC-ID / Instituto Superior Técnico). In the paper, this model is Table III → FT → WhisperLv3-X, trained on EP-425.

Results on the CAMÕES benchmark (WER %, from Table III)

The baselines are the public openai/whisper-large-v3 used as released (zero-shot rows of Table III), with plain Whisper decoding and with WhisperX (voice-activity detection before transcription), respectively.

EP domains: RS = read speech, BN = broadcast news, T/L = talks/lectures, CS = conversational speech, SI = sociolinguistic interviews. EP Avg. gives each domain equal weight. AAP = African and Asian Portuguese, BP = Brazilian Portuguese. All results use the CAMÕES text normaliser (lower-case, no punctuation, numbers spelled out). Leaderboard and test sets: inesc-id/camoes_asr.

Model details

• Base model: openai/whisper-large-v3 (1.55B parameters, all fine-tuned)
• Training data: EP-425, 425 h of European Portuguese from 14 corpora (Section III-A)
• Fine-tuning: 10 epochs, batch size 64, gradient accumulation 4, learning rate 1e-5, 10% warm-up (Section V)
• Decoding: WhisperX (voice-activity detection + batched faster-whisper), which reduces Whisper hallucinations
• Format: CTranslate2 (model.bin), float16 recommended

Usage

import whisperx

model = whisperx.load_model("inesc-id/WhisperLv3-EP-X", "cuda", compute_type="float16",
                            language="pt", task="transcribe")
audio = whisperx.load_audio("audio.wav")
result = model.transcribe(audio, batch_size=8, language="pt", task="transcribe")
print(" ".join(seg["text"] for seg in result["segments"]))

Output format

Pre-Acordo Ortográfico. All CAMÕES models are trained on transcripts written in the spelling from before the 1990 Orthographic Agreement (pré-Acordo), e.g. projecto, director, actual.

Transcriptions are normalised like the training transcripts: lower-case, no punctuation, and numbers written out in words.

Other CAMÕES models

Citation

@inproceedings{camoes,
  title     = {{CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese}},
  author    = {Carvalho, Carlos and Teixeira, Francisco and Botelho, Catarina and Pompili, Anna and
               Solera-Ure{\~n}a, Rub{\'e}n and Paulo, S{\'e}rgio and Juli{\~a}o, Mariana and Rolland, Thomas and
               Mendon{\c{c}}a, John and Pereira, Diogo and Trancoso, Isabel and Abad, Alberto},
  booktitle = {Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year      = {2025}
}