Downloads · 30 days
0
Jianshu001/wavlm-phoneme-scorer
wavlm-phoneme-scorer is a audio classification model from Jianshu001. Use it for the audio classification task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Fine-tuned WavLM-Large backbone + MLP scoring head for phoneme-level English pronunciation assessment, trained on 11K children's speech samples.
Downloads · 30 days
0
Access
Public
Updated Mar 31, 2026
Repo size
5.7 GB
Likes
2
Public
Click a slice to open those files.
.pt2.5 GB · 100%
From the Hugging Face model README
Fine-tuned WavLM-Large backbone + MLP scoring head for phoneme-level English pronunciation assessment, trained on 11K children's speech samples.
Given a reference text and an audio recording, this model identifies which phonemes are pronounced correctly and which have errors.
pip install torch torchaudio transformers g2p-en huggingface_hub
from pipeline_v2 import PronunciationAssessorV2
# Auto-download model from HuggingFace
assessor = PronunciationAssessorV2.from_pretrained()
result = assessor.assess("audio.mp3", "Hello, Peter.")
print(result["overall_score"]) # 85.2
print(result["n_errors"]) # 0
for word in result["words"]:
print(f"{word['word']}: score={word['score']}")
for ph in word["phonemes"]:
err = " <- ERROR" if ph["error"] else ""
print(f" /{ph['phone']}/ score={ph['score']} pherr={ph['pherr_prob']:.2f}{err}")
# Model downloads automatically on first run
python pipeline_v2.py --audio audio.mp3 --text "Hello, Peter."
Output:
============================================================
Text: "Hello, Peter."
Overall Score: 85.2/100 (errors: 0/8)
============================================================
✓ Hello score= 87.7 errors=0/4
/hh / score= 98.6 GOP= -0.97 pherr=0.05
/ah / score= 73.1 GOP= -7.40 pherr=0.43
/l / score= 88.6 GOP= +4.00 pherr=0.29
/ow / score= 90.7 GOP= -6.05 pherr=0.13
✓ Peter score= 82.6 errors=0/4
/p / score= 95.7 GOP= +5.40 pherr=0.08
/iy / score= 90.2 GOP= +3.70 pherr=0.12
/t / score= 72.5 GOP= +0.50 pherr=0.55
/er / score= 71.8 GOP= -1.40 pherr=0.61
# Via huggingface-cli
huggingface-cli download Jianshu001/wavlm-phoneme-scorer wavlm_finetuned.pt --local-dir .
# Via Python
from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="Jianshu001/wavlm-phoneme-scorer", filename="wavlm_finetuned.pt", local_dir=".")
# Then use with local path
assessor = PronunciationAssessorV2(checkpoint_path="wavlm_finetuned.pt")
Reference Text ──→ G2P ──→ Expected phoneme sequence
│
Audio ──→ CTC model ──→ Viterbi Forced Alignment ──→ Frame segments
│ │
└──→ WavLM-Large (fine-tuned) ──→ Hidden states ──→ Pool per segment
│
+ phone embedding (32d)
+ GOP score (1d)
+ n_frames (1d)
│
MLP (1058 → 512 → 512 → 256)
├── score_head → phoneme score (0-100)
└── pherr_head → error probability (0-1)
facebook/wav2vec2-xlsr-53-espeak-cv-ft (frozen, CTC-based Viterbi forced alignment)Evaluated on test set (8062 phonemes from 1727 audio files, children's speech):
| Metric | GOP Baseline (v1.0) | This Model |
|---|---|---|
| Phoneme Error AUC-ROC | 0.738 | 0.870 |
| Phoneme Error F1 | 0.476 | 0.595 |
| Phoneme Error Precision | 0.379 | 0.592 |
| Phoneme Error Recall | 0.638 | 0.598 |
| Phone Score Pearson | 0.372 | 0.645 |
| Phone Score MAE | 27.44 | 16.47 |
| File | Description | Size |
|---|---|---|
wavlm_finetuned.pt | Full checkpoint (backbone + head state dict) | 1.2GB |
pipeline_v2.py | Inference pipeline with from_pretrained() support | 18KB |
finetune_wavlm.py | Training script (reproducing the fine-tuning) | 25KB |
See the complete project (data, evaluation, all experiments) at: github.com/Jianshu-She/Voice-correction (branch: feature/wavlm-pipeline)