buddy-pronunciation-onnx

buddy-pronunciation-onnx

int8 ONNX exports of two phoneme-recognition models, for on-device pronunciation assessment in Buddy.

These are format conversions only — no retraining, no fine-tuning. See NOTICE for the exact export and quantization steps, as Apache-2.0 requires.

Contents

Each sits beside its vocab.json, copied verbatim from the source repository.

Paths are versioned and immutable. A future export goes to v2/; files already published are never replaced, so a client that downloaded v1 keeps exactly what it verified.

Why two models

They were chosen by measurement, not by reputation. Both were run against the same clips — clean speech, a planted /v/→/w/ substitution, and accented speech — and scored against CMUdict:

The English model was preferred over an L2-trained alternative that scored worse, and over a smaller 123 MB model that carries no stress information.

The Mandarin model mis-hears English think as s ɪ ŋ k — exactly the substitution a learner makes — so it is not used for English, where a false accusation would be worse than a missed error.

Usage

Both are CTC models. Feed 16 kHz mono float32 audio, normalized to zero mean and unit variance (what wav2vec2 was trained on — skipping this does not error, it quietly degrades every output):

import numpy as np, soundfile as sf, onnxruntime as ort, json

audio, sr = sf.read("speech.wav")          # 16 kHz mono
audio = audio.astype(np.float32)
audio = (audio - audio.mean()) / (audio.std() + 1e-7)

sess = ort.InferenceSession("en/v1/model.int8.onnx")
logits = sess.run(None, {"input_values": audio[None, :]})[0]   # [1, frames, vocab]

vocab = {v: k for k, v in json.load(open("en/v1/vocab.json")).items()}
ids = logits.argmax(-1)[0]
out, prev = [], None
for i in ids:                               # collapse CTC repeats and blanks
    if i != prev and i != 0:
        out.append(vocab[int(i)])
    prev = i
print(" ".join(out))

The output is logits, not probabilities — apply log-softmax per frame before using them as posteriors.

Measured at ~77 ms for a 2.3 s clip on CPU, roughly 30× faster than real time.

Licence

Apache-2.0, inherited from both originals. Cite the original authors rather than this repository; it exists only to host a runnable format.