VaaniCall

<br>

VaaniCall

Call-grade speech recognition for 11 Indian languages.
By Tausif Iqbal. Built on NVIDIA NeMo FastConformer, fine-tuned from VAANI for PSTN, VoIP, and contact-center audio.

VaaniCall transcribes a phone call without a language code. It infers language and script, and it is trained through a 21-stage telephony pipeline so AMR, packet loss, and handset noise look like training data — not a domain shift.

---

Why VaaniCall

---

Performance at a glance

All numbers are Word Error Rate (lower is better). Evaluations use 500 clips per language.

Whisper large-v3 still leads on English. VaaniCall is the Indic-language model.

---

Benchmarks

vs Whisper large-v3 — IndicVoices, telephonic

<details> <summary><b>IndicVoices · telephonic · full table</b></summary> </details> <details> <summary><b>IndicVoices · clean · full table</b></summary> </details>

vs Whisper large-v3 — Google FLEURS (out of domain)

Held-out read speech. Odia is not in FLEURS.

<details> <summary><b>FLEURS · clean · full table</b></summary>

Tamil CER is lower for Whisper (12.11%) despite a higher WER — typically an artifact of shorter predicted strings. VaaniCall still wins WER.

</details>

Per-language profile

Strongest Indic languages on phone audio: Punjabi 14.5% · Bengali 15.4% · Hindi 16.0% · Marathi 17.1%.

vs VAANI base — the telephony fine-tune

VaaniCall is a telephony specialist on top of VAANI. On phone-channel audio it improves 9 of 10 Indic languages. On clean audio the two models are statistically tied (24.37% vs 24.38% WER).

<details> <summary><b>vs VAANI base · telephonic</b></summary>

Δ = VAANI − VaaniCall. Positive Δ → VaaniCall is better. English excluded (not a VAANI fine-tune target).

</details> <details> <summary><b>vs VAANI base · clean</b></summary>

Results are statistically equivalent on clean audio — the fine-tune does not spend general accuracy to buy telephony robustness.

</details>

vs AI4Bharat IndicConformer

IndicConformer is a strong general-purpose Indic ASR and still leads this comparison. VaaniCall is optimized for call-channel robustness and zero-shot language ID, not for matching a dedicated per-language conformer on clean read speech.

Largest telephonic gaps: Malayalam −5.61 pp, Telugu −5.49 pp. English excluded.

<details> <summary><b>vs IndicConformer · telephonic</b></summary> </details> <details> <summary><b>vs IndicConformer · clean</b></summary> </details>

---

How it works

Training

Two-phase fine-tune from the VAANI FastConformer checkpoint.

1. Language alignment — encoder frozen; decoder learns script and language mapping.
2. Acoustic refinement — last five encoder layers plus the full decoder unfrozen for telephonic acoustics.

Data

Trained on AI4Bharat IndicVoices and VAANI: 50,000 train / 5,000 val / 5,000 test clips per language (mean duration ~18 s).

Telephony augmentation

Every training batch can pass through a 21-stage, GPU-accelerated call-channel simulator.

<details> <summary><b>All 21 stages</b></summary> </details>

---

Quick start

from nemo.collections.asr.models import EncDecRNNTBPEModel

model = EncDecRNNTBPEModel.from_pretrained(
    model_name="TieIncred/VaaniCall"
)

hypotheses = model.transcribe(["sample.wav"], return_hypotheses=True)
print(hypotheses[0].text)

No language flag. 16 kHz mono WAV is the happy path; typical telephony (8 kHz, μ-law / AMR) is the training domain.

---

Intended use

• Inbound / outbound contact-center transcription in Indic languages
• VoIP and PSTN call analytics
• Multilingual IVR and voice-bot post-processing
• Offline batch transcription of call recordings

Not intended for: medical or legal dictation, real-time emergency dispatch, or English-only broadcast ASR (use Whisper or a dedicated English model).

---

Limitations

• English trails Whisper large-v3 (expected: this checkpoint is Indic-first).
• Malayalam, Kannada, Tamil remain the hardest languages (WER 34–41% on IndicVoices).
• Odia slightly regresses vs the VAANI base under telephony (−1.37 pp).
• IndicConformer is still stronger as a general Indic ASR.
• Code-mixed utterances and heavy music-on-hold are not separately benchmarked.
• Evaluations are 500 clips / language; treat per-language gaps of <1 pp as noise.

---

Citation

If you use VaaniCall, please cite this model and VAANI, the base the fine-tune starts from:

@misc{iqbal2026vaanicall,
  title={VaaniCall: Multilingual Telephony ASR for Indic Languages},
  author={Iqbal, Tausif},
  year={2026},
  howpublished={Hugging Face},
  url={https://huggingface.co/TieIncred/VaaniCall}
}

@misc{pulikodan2026vaanicapturinglanguagelandscape,
  title={VAANI: Capturing the language landscape for an inclusive digital India},
  author={Sujith Pulikodan and Abhayjeet Singh and Agneedh Basu and Nihar Desai and Pavan Kumar J and Pranav D Bhat and Raghu Dharmaraju and Ritika Gupta and Sathvik Udupa and Saurabh Kumar and Sumit Sharma and Vaibhav Vishwakarma and Visruth Sanka and Dinesh Tewari and Harsh Dhand and Amrita Kamat and Sukhwinder Singh and Shikhar Vashishth and Partha Talukdar and Raj Acharya and Prasanta Kumar Ghosh},
  year={2026},
  eprint={2603.28714},
  archivePrefix={arXiv},
  primaryClass={eess.AS},
  url={https://arxiv.org/abs/2603.28714}
}

Also cite IndicVoices when reporting numbers on that test set.

---

<p align="center"> <em>VaaniCall · Tausif Iqbal · Apache 2.0</em> </p>