childvox-speechmaturity-whisper-large

Whisper-Large for Child Speech Maturity Classification

Model Description

This model includes the implementation of child speech maturity classification described in <a href="https://arxiv.org/abs/2605.29257"><strong>ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood</strong></a> (Accepted to EMNLP 2026 Main)

Github repository: https://github.com/tiantiaf0627/childvox-release

The model is fine-tuned on the SpeechMaturity dataset, a large-scale corpus of child vocalizations.

The included child vocalization categories are:

[
  'Canonical',
  'Non-Canonical',
  'Crying',
  'Laughing',
  'Junk'
]

Canonical denotes mature syllables containing a consonant-vowel transition, while Non-Canonical denotes immature vocalizations such as isolated vowels or consonants. Junk covers segments that are not child vocalizations (e.g., noise, adult speech, or unintelligible audio).

How to use this model

Download repo

git clone [email protected]:tiantiaf0627/childvox-release

Install the package

conda create -n childvox python=3.10
cd childvox
pip install -e .

Load the model

# Load libraries
import torch
import torch.nn.functional as F
from src.model.childvox.whisper_audio import WhisperWrapper

# Find device
device = torch.device("cuda") if torch.cuda.is_available() else "cpu"

# Load model from Huggingface
# We provide model with different folds, and specify the fold from 1, 2, 3, 4, 5
model = WhisperWrapper.from_pretrained("tiantiaf/childvox-speechmaturity-whisper-large", fold_idx=1).to(device)
model.eval()

Prediction

# Label List
maturity_list = [
  'Canonical',
  'Non-Canonical',
  'Crying',
  'Laughing',
  'Junk'
]

# Load data, here just zeros as the example
# The child vocalization segments used in training are short, so we cap the input at 1 seconds
# You need to prepare your audio to a length of 1 seconds, 16kHz and mono channel
max_audio_length = 1 * 16000
data = torch.zeros([1, 160000]).float().to(device)[:, :max_audio_length]
logits, embeddings = model(data, return_feature=True)

# Probability and output
maturity_prob = F.softmax(logits, dim=1)
print(maturity_list[torch.argmax(maturity_prob).detach().cpu().item()])

Responsible Use: Child speech data is highly sensitive. Users should respect the privacy and consent of the children and families whose recordings are processed, obtain approval from the appropriate ethics/IRB body, and adhere to the relevant laws and regulations in their jurisdictions when using ChildVox.

If you have any questions, please contact: Tiantian Feng ([email protected])

❌ Out-of-Scope Use

• Clinical or diagnostic applications (e.g., screening for developmental or language disorders)
• Individual-level developmental assessment without expert human review
• Surveillance
• Privacy-invasive applications
• No commercial use

If you like our work or use the models in your work, kindly cite the following. We appreciate your recognition!

@article{feng2026childvox,
  title={ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood},
  author={Feng, Tiantian and Xu, Anfeng and Shi, Xuan and Kommineni, Aditya and Siam, Shakhrul Iman and Micheletti, Megan and Shi, Zhonghao and Tager-Flusberg, Helen and Zhang, Mi and Perry, Lynn K and others},
  journal={arXiv preprint arXiv:2605.29257},
  year={2026}
}

You should also cite the following dataset if you use this model:

@inproceedings{zhang25r_interspeech,
  title     = {{Employing self-supervised learning models for cross-linguistic child speech maturity classification}},
  author    = {Theo Zhang and Madurya Suresh and Anne Warluamont and Kasia Hitczenko and Alejandrina Cristia and Margaret Cychosz},
  year      = {2025},
  booktitle = {{Interspeech 2025}},
  pages     = {2825--2829},
  doi       = {10.21437/Interspeech.2025-1946},
  issn      = {2958-1796},
}