childvox-speechocean762-accuracy-whisper-large

Whisper-Large for Child Speech Accuracy Classification

Model Description

This model includes the implementation of speech accuracy classification (half-children and half-adult) described in <a href="https://arxiv.org/abs/2605.29257"><strong>ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood</strong></a> (Accepted to EMNLP 2026 Main)

Github repository: https://github.com/tiantiaf0627/childvox-release

The model is fine-tuned on the SpeechOcean762 dataset, a large-scale corpus of speech dataset with quality assessment (half children, half adult).

The included prosody categories are:

[
  "Poor or Understandable",  # <= 6 from original scores
  "Good",                    # 7-8 from original scores
  "Excellent"                # >8 from original scores
]

How to use this model

Download repo

git clone [email protected]:tiantiaf0627/childvox-release

Install the package

conda create -n childvox python=3.10
cd childvox
pip install -e .

Load the model

# Load libraries
import torch
import torch.nn.functional as F
from src.model.childvox.whisper_audio import WhisperWrapper

# Find device
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"

# Load model from Huggingface
# We provide model with different folds, and specify the fold from 1, 2, 3, 4, 5
model = WhisperWrapper.from_pretrained("tiantiaf/childvox-speechocean762-accuracy-whisper-large", fold_idx=1).to(device)
model.eval()

Prediction

# Label List
accuracy_list = [
  "Poor or Understandable",  # <= 6 from original scores
  "Good",                    # 7-8 from original scores
  "Excellent"                # >8 from original scores
]

# Load data, here just zeros as the example
# The child speech segments used in training, which we cap the input at 10 seconds (even the model specify 15 seconds)
# You need to prepare your audio to a length of 10 seconds, 16kHz and mono channel
max_audio_length = 10 * 16000
data = torch.zeros([1, 160000]).float().to(device)[:, :max_audio_length]
logits, embeddings = model(data, return_feature=True)

# Probability and output
accuracy_prob = F.softmax(logits, dim=1)
print(accuracy_list[torch.argmax(accuracy_prob).detach().cpu().item()])

Responsible Use: Child speech data is highly sensitive. Users should respect the privacy and consent of the children and families whose recordings are processed, obtain approval from the appropriate ethics/IRB body, and adhere to the relevant laws and regulations in their jurisdictions when using ChildVox.

If you have any questions, please contact: Tiantian Feng ([email protected])

❌ Out-of-Scope Use

• Clinical or diagnostic applications (e.g., screening for developmental or language disorders)
• Individual-level developmental assessment without expert human review
• Surveillance
• Privacy-invasive applications
• No commercial use

If you like our work or use the models in your work, kindly cite the following. We appreciate your recognition!

@article{feng2026childvox,
  title={ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood},
  author={Feng, Tiantian and Xu, Anfeng and Shi, Xuan and Kommineni, Aditya and Siam, Shakhrul Iman and Micheletti, Megan and Shi, Zhonghao and Tager-Flusberg, Helen and Zhang, Mi and Perry, Lynn K and others},
  journal={arXiv preprint arXiv:2605.29257},
  year={2026}
}

Please also cite the following dataset used to develop this model

@inproceedings{zhang21x_interspeech,
  title     = {{speechocean762: An Open-Source Non-Native English Speech Corpus for Pronunciation Assessment}},
  author    = {Junbo Zhang and Zhiwen Zhang and Yongqing Wang and Zhiyong Yan and Qiong Song and Yukai Huang and Ke Li and Daniel Povey and Yujun Wang},
  year      = {2021},
  booktitle = {{Interspeech 2021}},
  pages     = {3710--3714},
  doi       = {10.21437/Interspeech.2021-1259},
  issn      = {2958-1796},
}