Downloads · 30 days
0
Jianshu001/wavlm-word-v13c2
wavlm-word-v13c2 is a audio classification model from Jianshu001. Use it for the audio classification task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Single-word pronunciation scoring model trained on 860K real user evaluations (children English learning). Given an audio clip and the expected word, predicts a 0–100 score that approximates an English-evaluation engi…
Downloads · 30 days
0
Access
Public
Updated May 22, 2026
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.pt1.3 GB · 100%
From the Hugging Face model README
Single-word pronunciation scoring model trained on 860K real user evaluations (children English learning). Given an audio clip and the expected word, predicts a 0–100 score that approximates an English-evaluation engine's overall output.
WavLM-Large (top 12 layers unfrozen) + phone embeddings as cross-attention queries + multi-task head:
audio (16 kHz mono) → WavLM-Large → audio_frames (B, T_a, 1024)
expected phone IDs (≤16 phones) → phone_emb (50 → 256 → 1024) → ph_q
ph_q × audio_frames (cross-attention) → per-phone repr (B, T_p, 1024)
per-phone repr → per-phone score head (regression, 0..100)
per-phone repr + audio attention pool → word score head (regression, 0..100)
Trained with stratified sampling over 5 score bands (0–20, 20–40, 40–60, 60–80, 80–100), Huber loss δ=5,
L = 1.0·L_word + 0.8·L_phone.
| Metric | V9 baseline | V13c2 |
|---|---|---|
| MAE (lower better) | 37.98 | 12.63 |
| Pearson corr | 0.665 | 0.819 |
| P(|err|<5) | 6.1% | 29.2% |
| P(|err|<10) | — | 51.6% |
| GT band | n | MAE | P<5 |
|---|---|---|---|
| 0–20 | 19,879 | 15.74 | 15.7% |
| 20–40 | 20,095 | 12.41 | 32.4% |
| 40–60 | 19,724 | 13.23 | 21.9% |
| 60–80 | 19,926 | 11.58 | 30.2% |
| 80–100 | 20,233 | 10.25 | 45.2% |
wavlm_word_v13c2.pt — model checkpoint (state_dict + phone_vocab + training args + metrics)train_v13_multitask.py — training script with model class definitionimport torch
import sys, os
# Make sure train_v13_multitask.py is on the path so the model class can be imported
sys.path.insert(0, "/path/to/checkpoint/directory")
from train_v13_multitask import V13Model
ckpt = torch.load("wavlm_word_v13c2.pt", map_location="cpu", weights_only=False)
phone_vocab = ckpt["phone_vocab"]
phone_to_id = {p: i for i, p in enumerate(phone_vocab)}
model = V13Model(n_phones=len(phone_vocab), unfreeze_top_n=12).eval().cuda()
model.load_state_dict(ckpt["model_state"])
# Prepare a single example:
import soundfile as sf, numpy as np, torch.nn.functional as F
wav, sr = sf.read("user_word.mp3", dtype="float32")
if wav.ndim > 1: wav = wav.mean(axis=1)
if sr != 16000:
idxs = np.linspace(0, len(wav)-1, int(len(wav)*16000/sr)).astype(np.int64)
wav = wav[idxs]
wav = wav[:40000] # truncate to 2.5s
wav = np.pad(wav, 4800, mode="constant") # 0.3s silence each side
wav = (wav - wav.mean()) / (wav.std() + 1e-7)
mask = np.ones_like(wav, dtype=np.float32)
# Expected phones for "street" (use your phone tokenizer; vocab listed in ckpt['phone_vocab'])
phones = ["s", "tr", "iy", "t"]
phone_ids = [phone_to_id.get(p, 1) for p in phones]
# Pad to MAX_PHONES=16
phone_ids += [0] * (16 - len(phone_ids))
ph_mask = [1.0] * len(phones) + [0.0] * (16 - len(phones))
with torch.no_grad():
word_pred, phone_pred = model(
torch.FloatTensor(wav).unsqueeze(0).cuda(),
torch.FloatTensor(mask).unsqueeze(0).cuda(),
torch.LongTensor([phone_ids]).cuda(),
torch.FloatTensor([ph_mask]).cuda(),
)
print(f"word score: {float(word_pred):.1f}")
print(f"per-phone scores: {phone_pred[0][:len(phones)].tolist()}")
48 IPA-derived phone symbols + 2 special tokens (<pad>, <unk>); full list is ckpt['phone_vocab'].
overall 0–100, per-phone scores)If you use this checkpoint:
@misc{she2026wavlmwordv13c2,
title = {WavLM Word V13c2: WavLM-Large + phone cross-attention for English pronunciation scoring},
author = {She, Jianshu},
year = {2026},
url = {https://huggingface.co/Jianshu001/wavlm-word-v13c2},
}