betterflow-bengali-streaming-transducer

betterflow — Bengali streaming transducer (sherpa-onnx)

Streaming zipformer2 transducer exported to ONNX for on-device dictation. Emits partials DURING speech, so first-partial latency is constant rather than proportional to utterance length — the property batch whisper structurally lacks.

• encoder.int8.onnx (24 MB) + decoder.onnx (2.0 MB) + joiner.onnx (0.98 MB) + tokens.txt (500 BPE tokens) + bpe.model — 27 MB total
• Load with sherpa-onnx OnlineRecognizer.from_transducer, model_type="zipformer2"
• Source weights: alphacep/vosk-model-small-streaming-bn, Apache-2.0, trained by Alphacephei (Vosk) with k2-fsa/icefall. Weights unmodified but for int8 quantization of the encoder.

Mirrored so betterflow can pin an immutable revision and checksum.

‼️ This is a TRANSDUCER, not CTC — four files, not two

Our Hindi bundle is a Zipformer2-CTC: encoder.int8.onnx + tokens.txt. This one needs encoder + decoder + joiner. ModelHandle.kindOf resolves by inspecting the file list, so the manifest partial.files array has four entries.

‼️ Only the ENCODER is int8-quantized. That is deliberate.

Quantizing decoder and joiner as well produces degenerate repetitive output. They are ~1–2 MB networks with almost no weights to quantize, and they sit in the autoregressive loop where error compounds token over token. The encoder is 86 MB of feed-forward weights and quantizes cleanly (86.78 MB → 24.08 MB, 3.6×). This matches the sherpa-onnx zoo convention.

‼️ The upstream fp32 encoder does not decode correctly; this int8 one does

The raw encoder.onnx returns near-empty output under sherpa-onnx 1.13.4 ("", আমার, ফুল), while the int8 graph built from it returns full coherent sentences. Upstream ships am-onnx/opt.py precisely because the raw export needs ONNX graph optimization; quantize_dynamic's rewriting does the equivalent. Do not "fix" this by reverting to fp32 — and do not read a fp32-vs-int8 comparison as evidence that quantization degraded anything.

Verified

Decoded through sherpa-onnx 1.13.4 (the version vendored in the betterflow client, so this exercises the engine that actually ships):

• 100% Bengali script across 3 clips, coherent rather than merely Bengali-shaped — one test clip returns the JFK line in full: তোমার দেশ তোমার জন্য কি করতে পারে তা জিজ্ঞেস করো না…
• Deterministic: byte-identical inputs produce byte-identical output.

‼️ Not verified

• WER is UNMEASURED. No Bengali eval set exists. Do not assume it matches our Hindi model.
• First-partial latency unmeasured.
• No punctuation. tokens.txt contains 500 tokens and zero punctuation — no danda (।), no comma, no question mark. Output is an unbroken run of words. This is not specific to Bengali: our shipped Hindi streaming model has zero punctuation too. Punctuation is a text-cleanup responsibility, not a model one.