gigaam-v3-e2e-rnnt-sherpa-onnx

GigaAM v3 e2e RNN-T — sherpa-onnx format

Russian ASR model packaged for sherpa-onnx, including on-device use on Android and iOS.

Unlike the officially converted GigaAM models, this is the end-to-end variant: it outputs punctuation, capitalization and normalized numbers directly, with no separate post-processing step.

без e2e:  так так а вот это влад это значит у тебя сейчас дима на когда в работе
с e2e:    Так. Так. А вот это, Влад, это, значит, а у тебя сейчас, Дим, на когда в работе?

Files

Usage

import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder.int8.onnx",
    decoder="decoder.onnx",
    joiner="joiner.onnx",
    tokens="tokens.txt",
    num_threads=2,
    sample_rate=16000,
    feature_dim=64,          # GigaAM uses 64 mel bins, not 80
    decoding_method="greedy_search",
    model_type="nemo_transducer",   # required: auto-detection does not trigger
)

Two settings are easy to get wrong and both are required: feature_dim=64 (most NeMo models use 80) and an explicit model_type.

Important: always segment long audio

The encoder has positional embeddings for 5000 frames — with 4× subsampling that is 200 seconds of audio per call, beyond which onnxruntime throws Attempting to broadcast an axis by a dimension other than 1.

Quality degrades long before that hard limit. Measured on real meeting recordings:

The model was trained on segments up to ~30 s. Feed it VAD-segmented chunks of about 25–30 s; anything longer silently loses most of the speech.

Benchmarks

Samsung Galaxy S25 Ultra, CPU, 2 threads, 30 s chunks:

More threads and NNAPI gave no speedup: the bottleneck is the sequential RNN-T decoder, not the encoder.

How this was made

Starting from istupakov/gigaam-v3-onnx, which targets the onnx-asr runtime, four changes were needed for sherpa-onnx:

1. Encoder metadata added (is_giga_am, vocab_size, pred_hidden, pred_rnn_layers, subsampling_factor) — sherpa-onnx identifies GigaAM by these.
2. Decoder input x cast from int64 to int32.
3. Decoder output dec transposed from [1, 1, 320] to [1, 320, 1].
4. Dummy unused_x_len input/output added, and inputs reordered to x, unused_x_len.1, h.1, c.1 — sherpa-onnx binds them positionally, not by name.

Conversion script: convert_e2e.py

Credits and license

MIT, inherited from the original model.

• Original model: GigaAM by SberDevices
• ONNX export: istupakov/gigaam-v3-onnx
• Runtime: sherpa-onnx by k2-fsa

This repository only repackages existing work; no retraining or fine-tuning was done.