betterflow-sravaani-streaming-onnx

Betterflow — SraVaani streaming, ONNX int8 for sherpa-onnx

An ONNX export of ARTPARK-IISc/SraVaani-0.5-live, prepared so it loads in sherpa_onnx.OnlineRecognizer and produces live partials for nine Indian languages.

We are not the authors of the weights. Upstream is ARTPARK-IISc; this repo is a format conversion plus quantization.

Provenance and licence

Please cite ARTPARK-IISc for the underlying model.

Why this export exists

Upstream ships fp16 TorchScript with trust_remote_code, which sherpa-onnx cannot load. torch.jit.load turns out to expose a literal NeMo ConformerEncoder (5-in/5-out) plus a ctc_decoder, and the shipped graph already carries sherpa's cache contract — so a single scripted fusing wrapper was enough.

Contents

model-la13.onnx    658,699,885 B   MatMul-only int8, att_context_size [70,13]
tokens.txt              68,907 B   5,001 entries = the model's output dim
export-la13.json                   export settings and provenance

Measured

vaani-{hi,gu}, n=50 each, through sherpa with a padded tail, same normalizer as our other evaluations:

The TorchScript parent scores 29.2 / 28.8 on the same clips, so export + sherpa costs under 1.5 points. Partials arrive incrementally and monotonically — they never rewrite:

'बहुत बड़े' → 'बहुत बड़े नहर' → 'बहुत बड़े नहर जो बह रही' → 'बहुत बड़े नहर जो बह रही है'

⚠️ Four things that will bite you

1. Pad the tail, or lose 14 WER points. sherpa never decodes the trailing partial chunk. After InputFinished(), push ~1,300 ms of silence before reading the final result. Unpadded hi scores 43.7%; padded, 29.7%.

2. Use [70,13], not [70,6]. The shorter lookahead is worse, not merely faster — hi 29.7 → 33.2, gu 30.2 → 36.3.

3. Normalization is folded in — do not strip metadata. With the wrong normalization this model loads, runs, and returns EMPTY with no error. Acceptance must be a decode producing the right script, never a successful load.

4. It CODE-SWITCHES, and a naive script-purity gate will reject it for that. Own-script share is 75.6–85.7%, but ~90% of the Latin on Hindi is real English words (bank, computer). If you gate on script purity, gate on the non-word fraction, or you will reject the behaviour that makes it useful for code-switched speech.

Cost, and one structural advantage

Peak RSS is FLAT in utterance length — 1,531 MB at 5 s, 1,575 MB at 134 s. A streaming model has bounded context and therefore bounded memory, unlike an offline Conformer whose attention is O(T²). No utterance-length cap is needed.

int8 is close to free here: 1.79 GB → 659 MB on disk for 29.7 → 29.6 WER.

Not evaluated

Device/Android verification · the [70,1] and [70,0] arms · languages beyond hi/gu through this export (the parent was scored on all nine) · dictation-register audio — everything above is read/prompted speech.