Shenava-Koochik-v1.5-RNNT-sherpa-onnx

🦻 Shenava · Koochik v1.5 — RNNT for sherpa-onnx

Persian (فارسی) on-device speech recognition. This is the RNNT / transducer branch of Reza2kn/Shenava-Koochik-v1.5 — a 114M FastConformer hybrid — exported as an offline transducer for sherpa-onnx (encoder + decoder + joiner).

It runs with the same binary everywhere sherpa-onnx does: Linux · macOS · Windows · Android · iOS · WebAssembly · Raspberry Pi, from Python, C++, C#, Go, Swift, Kotlin, Dart, Rust, and more.

Part of VisualEars / Shenava — open, offline, real-time Persian captioning for the Deaf & Hard-of-Hearing.

---

✨ Why this model exists

v1.0's RNNT head was broken — a tokenizer/blank misalignment sent greedy decoding into an infinite loop (~4398 % WER, unusable). v1.5 revives it: the prediction network and joint are surgically re-initialised under the ve_tok_v4 tokenizer and trained up with a two-phase curriculum → a genuinely working transducer head. Crucially, the record-setting CTC head is frozen and preserved bit-for-bit during the corrective fine-tune, so there is zero CTC regression.

This repository packages the transducer branch only for sherpa-onnx. If you want the lower-WER CTC head, see the related models below.

📊 Benchmarks

golden-6669 — a held-out Persian test set (strict fair_text normalizer, att_context [70,13]):

The RNNT head trails the CTC head slightly on WER but is a fully independent decoding path (useful for fusion, confidence, or streaming experiments).

📦 What's inside

Both precisions are included — pick per your size/latency budget:

int8 is ~3.5× smaller and decoded byte-identically to fp32 on our Persian test clips — it's a free win for most deployments.

🚀 Quick start

Python

pip install sherpa-onnx soundfile

import sherpa_onnx, soundfile as sf

recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder.onnx",          # or encoder.int8.onnx
    decoder="decoder.onnx",          # or decoder.int8.onnx
    joiner="joiner.onnx",            # or joiner.int8.onnx
    tokens="tokens.txt",
    model_type="nemo_transducer",    # ⚠️ REQUIRED — see notes below
    num_threads=2,
    sample_rate=16000,
    feature_dim=80,
    decoding_method="greedy_search",
)

audio, sr = sf.read("audio.wav", dtype="float32")   # 16 kHz mono
if audio.ndim > 1:
    audio = audio[:, 0]

stream = recognizer.create_stream()
stream.accept_waveform(sr, audio)
recognizer.decode_stream(stream)
print(stream.result.text)

Command line

sherpa-onnx-offline \
  --encoder=encoder.int8.onnx \
  --decoder=decoder.int8.onnx \
  --joiner=joiner.int8.onnx \
  --tokens=tokens.txt \
  --model-type=nemo_transducer \
  audio.wav

Example output (colloquial, see notes): سلام حال شما خوبه نه همچنان

⚠️ Important notes

• model_type="nemo_transducer" is required. Without it, sherpa-onnx routes to the icefall stateless-decoder path and fails with 'context_size' does not exist in the metadata. NeMo uses an LSTM prediction network, not a stateless context-N decoder.
• Un-normalized features. These models train with normalize=NA (they consume raw log-mel). The encoder metadata sets normalize_type="" so sherpa-onnx does not re-normalize — otherwise the output is garbage. This is baked into the model; you don't need to do anything.
• Full-context / offline. The encoder is exported at att_context [-1,-1] for offline (whole-utterance) decoding, matching the offline CTC package. For real-time streaming, see the streaming export recipe linked below.
• Colloquial output & numbers. The RNNT head is trained on audio-faithful labels, so it leans colloquial and drops sentence punctuation (؟ / ،). Numbers are emitted in spoken form (هشت, not ۸) — apply Persian inverse text normalization at display time if you need digits.

🧠 Model details

🔁 Reproduce

export-rnnt-sherpa.py (included) loads the v1.5 .nemo, sets full context, and exports the transducer branch following the official sherpa-onnx NeMo recipe:

python export-rnnt-sherpa.py shenava-koochik-v1.5.nemo

🔗 Related models

📄 License & citation

Released under the Apache License 2.0. If you use it:

@misc{shenava_koochik_v15_rnnt_sherpa,
  title  = {Shenava Koochik v1.5 — RNNT for sherpa-onnx},
  author = {Sayar, Reza},
  year   = {2026},
  url    = {https://huggingface.co/Reza2kn/Shenava-Koochik-v1.5-RNNT-sherpa-onnx}
}

Built with ❤️ for accessibility · shenava.app