nb-whisper-dialect-id-5dialect

nb-whisper-dialect-id-5dialect

A NbAiLab/nb-whisper-medium encoder fine-tuned for 5-class Norwegian dialect identification.

Given an audio clip of Norwegian speech, the model predicts which of five broad dialect regions the speaker belongs to:

Results from this model are discussed in further detail in:

Phoebe Parsons, Per Erik Solberg, Knut Kvale, Torbjørn Svendsen, and Giampiero Salvi. (2025). Adding Metadata to Existing Parliamentary Speech Corpus. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), pages 448–457, Tallinn, Estonia. University of Tartu Library.

Model description

The model is NbAiLab/nb-whisper-medium's encoder with a classification head on top (WhisperForAudioClassification), fully fine-tuned (no layers frozen) for sequence classification over 5 dialect labels.

Intended uses & limitations

Intended for research on Norwegian dialect identification — e.g. as a component in dialect-aware speech pipelines, or as a baseline for further dialect-ID research. It is not intended for consequential decisions about individuals (e.g. hiring, legal, or identity-verification contexts).

Known limitations:

• Trained only on adult parliamentary speech (see below); accuracy on other domains (conversational speech, children, non-native speakers) is not guaranteed. Separate evaluations of this model on read-speech (NB Tale) and human-survey audio are available in the source repo's results_dir/.
• The final training epoch showed signs of overfitting (see Training results below); this repo ships the best checkpoint by eval accuracy, not the last chronological one.
• The 5 dialect categories are coarse regional groupings, not a fine-grained dialect taxonomy, and boundaries between adjacent regions are inherently fuzzy.

How to use

import torch
from transformers import AutoFeatureExtractor, AutoModelForAudioClassification
import librosa

model_id = "scribe-project/nb-whisper-dialect-id-5dialect"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
model = AutoModelForAudioClassification.from_pretrained(model_id)

audio, sr = librosa.load("path/to/audio.wav", sr=feature_extractor.sampling_rate, mono=True)

inputs = feature_extractor(
    audio,
    sampling_rate=feature_extractor.sampling_rate,
    return_tensors="pt",
)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_id = torch.argmax(logits, dim=-1).item()
print(model.config.id2label[predicted_id])

Audio should be mono, resampled to 16kHz. Clips longer than 30 seconds were randomly subsampled to 30 seconds during training.

Training and evaluation data

Trained on the SSC (Storting/Parliament Speech Corpus), using a speaker-disjoint train/validation split (no speaker overlap between train and eval). Each example is a single-speaker audio segment labeled with one of the 5 dialect categories above. The assignment of dialect labels are discussed in more detail in Parsons et al.

Training procedure

Training hyperparameters

• learning_rate: 3e-05
• train_batch_size: 16 (per device)
• eval_batch_size: 32 (per device)
• seed: 0
• distributed_type: multi-GPU
• num_devices: 6
• gradient_accumulation_steps: 2
• total_train_batch_size: 192
• optimizer: Adam (betas=(0.9, 0.999), epsilon=1e-08)
• lr_scheduler_type: linear, warmup_ratio 0.1
• num_epochs: 3.0
• mixed_precision_training: fp16 (DeepSpeed)
• feature encoder: not frozen (full fine-tune)
• model selection: best checkpoint by eval accuracy (load_best_model_at_end)

Training results

The weights published in this repo are from epoch 2 (checkpoint-906) — the best-performing checkpoint — not the final epoch-3 checkpoint, which had started to overfit (training loss kept falling while validation accuracy dropped).

Framework versions

• Transformers 4.40.0
• PyTorch 2.2.2+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1