whisper-small-bhojpuri-merged

Whisper-small Bhojpuri (merged)

openai/whisper-small with a Bhojpuri LoRA adapter already merged in. Standalone weights — no PEFT dependency, no adapter loading step.

The LoRA adapter, full training details, evaluation methodology, and the leakage audit live in the adapter repo: Aditya109/whisper-small-bhojpuri-lora. Read that one if you want to know how these numbers were produced.

Results

Official Vaani Bhojpuri test split, n = 1,426. Every row measured on the same utterances, with the same normalizer and the same metric implementation.

Against SraVaani — the model trained by the dataset's own authors — this is a split decision, and both halves are statistically significant under a paired bootstrap: SraVaani is better on WER by 1.61 (95% CI [+0.83, +2.44]), this model is better on CER by 3.24 (95% CI [−3.87, −2.53]).

Usage

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

REPO = "Aditya109/whisper-small-bhojpuri-merged"

processor = WhisperProcessor.from_pretrained(REPO, language="hi", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, attn_implementation="sdpa").to("cuda").eval()

feats = processor.feature_extractor(
    audio_16k, sampling_rate=16_000, return_tensors="pt"
).input_features.to("cuda", dtype=torch.bfloat16)

ids = model.generate(feats, max_new_tokens=200, num_beams=1)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Weights are stored in float32 so you can cast to whatever your hardware wants.

Notes

• Audio must be 16 kHz mono.
• The language token is hi. Whisper has no Bhojpuri token; Hindi is the nearest proxy, and the dialect knowledge is in the fine-tuned weights. The decoding defaults are baked into generation_config, so it transcribes Bhojpuri out of the box rather than language-detecting into Hindi.
• Being a plain Whisper checkpoint, this converts for faster-whisper, whisper.cpp and WhisperX. That portability is the main argument for it over a NeMo Conformer.
• Evaluated with greedy decoding on utterance-level clips. Long-form audio and beam search are untested.

Data and limitations

Trained on ARTPARK-IISc/Vaani-transcription-part (config Bhojpuri), ~24 hours of spontaneous image-description speech, using the dataset's official splits unmodified.

Vaani has no speaker ID column, so speaker-disjointness is trusted from the official splits, not verified — by me or by anyone else working from the public dataset. Text-level leakage was audited and ruled out; details and the audit script are in the adapter repo.

Other limitations: no standard-Hindi regression check was run, so the model may have degraded on standard Hindi; domain coverage is limited to one collection protocol; and some of the residual error is annotation noise in the references rather than model error.

Licensing

• Base model openai/whisper-small is MIT.
• Training data (Vaani) is CC-BY-4.0, which requires attribution, so this model is released under CC-BY-4.0. No Vaani audio or transcripts are redistributed here.

Attribution for the dataset, as requested on its dataset page:

@misc{pulikodan2026vaanicapturinglanguagelandscape,
      title={VAANI: Capturing the language landscape for an inclusive digital India},
      author={Sujith Pulikodan and Abhayjeet Singh and Agneedh Basu and Nihar Desai and Pavan Kumar J and Pranav D Bhat and Raghu Dharmaraju and Ritika Gupta and Sathvik Udupa and Saurabh Kumar and Sumit Sharma and Vaibhav Vishwakarma and Visruth Sanka and Dinesh Tewari and Harsh Dhand and Amrita Kamat and Sukhwinder Singh and Shikhar Vashishth and Partha Talukdar and Raj Acharya and Prasanta Kumar Ghosh},
      year={2026},
      eprint={2603.28714},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2603.28714}
}