WhisperLv3-X-PT-All

WhisperLv3-X-PT-All: Whisper large-v3 fine-tuned for all Portuguese varieties (WhisperX) — pré-Acordo

Whisper large-v3 fine-tuned on European, Brazilian, and African and Asian Portuguese (EP-425 + BP-417 + AAP-8), packaged for WhisperX. Use this model when the variety is unknown or mixed: it keeps EP performance close to the EP-only model while matching the best BP model.

From the paper CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese (INESC-ID / Instituto Superior Técnico). In the paper, this model is Table III → PT-All → WhisperLv3-X, trained on EP-425 + BP-417 + AAP-8.

Results on the CAMÕES benchmark (WER %, from Table III)

The baselines are the public openai/whisper-large-v3 used as released (zero-shot rows of Table III), with plain Whisper decoding and with WhisperX (voice-activity detection before transcription), respectively.

EP domains: RS = read speech, BN = broadcast news, T/L = talks/lectures, CS = conversational speech, SI = sociolinguistic interviews. EP Avg. gives each domain equal weight. AAP = African and Asian Portuguese, BP = Brazilian Portuguese. All results use the CAMÕES text normaliser (lower-case, no punctuation, numbers spelled out). Leaderboard and test sets: inesc-id/camoes_asr.

Model details

• Base model: openai/whisper-large-v3 (1.55B parameters, all fine-tuned)
• Training data: EP-425 (European), BP-417 (Brazilian: CETUC + CORAA train), AAP-8 (African and Asian: PoSTPort)
• Fine-tuning: same settings as the EP-only model (Section V)
• Decoding: WhisperX (voice-activity detection + batched faster-whisper)
• Format: CTranslate2 (model.bin), float16 recommended

Usage

import whisperx

model = whisperx.load_model("inesc-id/WhisperLv3-X-PT-All", "cuda", compute_type="float16",
                            language="pt", task="transcribe")
audio = whisperx.load_audio("audio.wav")
result = model.transcribe(audio, batch_size=8, language="pt", task="transcribe")
print(" ".join(seg["text"] for seg in result["segments"]))

Output format

Pre-Acordo Ortográfico. All CAMÕES models are trained on transcripts written in the spelling from before the 1990 Orthographic Agreement (pré-Acordo), e.g. projecto, director, actual.

Transcriptions are normalised like the training transcripts: lower-case, no punctuation, and numbers written out in words.

Other CAMÕES models

Citation

@inproceedings{camoes,
  title     = {{CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese}},
  author    = {Carvalho, Carlos and Teixeira, Francisco and Botelho, Catarina and Pompili, Anna and
               Solera-Ure{\~n}a, Rub{\'e}n and Paulo, S{\'e}rgio and Juli{\~a}o, Mariana and Rolland, Thomas and
               Mendon{\c{c}}a, John and Pereira, Diogo and Trancoso, Isabel and Abad, Alberto},
  booktitle = {Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year      = {2025}
}