Downloads · 30 days
0
Sarphix/canary-1b-v2-sherpa-onnx-int8
canary-1b-v2-sherpa-onnx-int8 is a automatic speech recognition model from Sarphix. Use it when you need speech turned into text. It is set up for onnx. The card lists the license as cc-by-4.0.
An sherpa-onnx offline-recognizer package for nvidia/canary-1b-v2, in the same encoder/decoder/tokens layout sherpa-onnx already ships for canary-180m-flash. Canary is an attention encoder-decoder model, not a transdu…
Downloads · 30 days
0
Access
Public
Updated Jul 11, 2026
Repo size
4.5 GB
Likes
0
Public
Click a slice to open those files.
.onnx3.4 GB · 75%
From the Hugging Face model README
An sherpa-onnx offline-recognizer
package for nvidia/canary-1b-v2,
in the same encoder/decoder/tokens layout sherpa-onnx already ships for
canary-180m-flash. Canary
is an attention encoder-decoder model, not a transducer, so there is no
joiner file.
This repo is a community conversion, not an official NVIDIA or k2-fsa
release. Starting from the .nemo checkpoint above, the following was done:
export_onnx_180m_flash.py
script, retargeted at nvidia/canary-1b-v2 instead of
nvidia/canary-180m-flash (export_canary_1b_v2.py, environment:
torch 2.6.0 CPU, nemo_toolkit[asr] 2.7.3, onnx 1.17.0, onnxruntime 1.17.1).onnxruntime.quantization.quantize_dynamic).encoder.int8.onnx,
decoder.int8.onnx, tokens.txt (no joiner), plus a test_wavs/ folder --
same structure as the official
sherpa-onnx-nemo-canary-180m-flash-en-es-de-fr-int8.tar.bz2 release.num_decoder_layers = 10 and
decoder_hidden_size = 1024. Note the semantics: the sherpa-onnx C++
runtime uses num_decoder_layers as the COUNT of decoder_mems_list_*
inputs it feeds. NeMo's TransformerDecoder keeps num_layers + 2 memory
states when a final_layer_norm exists (canary-1b-v2: 8 layers + final
LN = 10). The runtime's default of 6 matches 180m-flash, not this model.Key non-obvious fixes needed vs. the 180m script (why k2-fsa/sherpa-onnx#1767 stayed unresolved):
dynamo=True. The decoder wrapper reads
the decode position via decoder_input_ids[0][-1].item(); the legacy
TorchScript exporter bakes that .item() in as a constant 0, freezing
the positional embedding -- the model then emits one correct token and
degenerates into repeating its own prompt. The dynamo exporter keeps it a
dynamic value.decoder_input_ids vs. the mems (torch
2.6's exporter rejects the 180m script's shared "num_tokens" symbol as a
constraint violation).tokens.txt built with tokenizer.id_to_piece(i), not ids_to_text([i]):
canary-1b-v2's CanaryBPETokenizer strips the leading space on single-id
detokenization, which would drop the ▁ word-boundary markers (output
would have no spaces between words).TransformerDecoder.forward(return_mems=True) returns a
2-tuple (mems, xatt_scores); older NeMo returned just the list.Verified with the sherpa-onnx (1.13.4) Python runtime on the bundled
test_wavs/: German ASR ("Alles hat ein Ende, nur die Wurst hat zwei."),
English ASR, and de<->en translation all decode correctly.
| File | Description |
|---|---|
sherpa-onnx-nemo-canary-1b-v2-int8.tar.bz2 | Full package: encoder, decoder, tokens, test_wavs/ |
encoder.int8.onnx | int8-quantized encoder (also inside the tar) |
decoder.int8.onnx | int8-quantized decoder (also inside the tar) |
encoder.fp16.onnx | fp16 encoder (see "fp16 variant" below) |
decoder.fp16.onnx | fp16 decoder (see "fp16 variant" below) |
tokens.txt | Vocabulary (also inside the tar) |
sha256sums.txt | Checksums, for verifying your download |
encoder.fp16.onnx / decoder.fp16.onnx are half-precision versions of the
same export (use the same tokens.txt). For CPU inference the int8 files
are the recommended choice (smaller, comparable speed); fp16 is mainly
interesting for CUDA execution providers.
Producing an fp16 pair that current sherpa-onnx actually loads required more
than a plain convert_float_to_float16 call, documented here for anyone
reproducing this:
onnxruntime.transformers.float16, keep_io_types=True so the I/O stays
float32), after file-based shape inference (infer_shapes_path) on the
2 GB encoder so cast nodes get inserted at fp16/fp32 boundaries.
value_info entries from the decoder afterwards (the
converter leaves entries typed float32 that contradict the retyped
LayerNormalization outputs; onnxruntime rejects the graph otherwise).SimplifiedLayerNormFusion
pass on inserted casts attached to Constant outputs shared across the 32
encoder layers ("Attempting to get index by a name which does not
exist: InsertedPrecisionFreeCast_...").Verified with sherpa-onnx 1.13.4 (Python) on the bundled test_wavs/:
German and English ASR both decode correctly from the fp16 pair.
Python API:
import sherpa_onnx
recognizer = sherpa_onnx.OfflineRecognizer.from_nemo_canary(
encoder="./sherpa-onnx-nemo-canary-1b-v2-int8/encoder.int8.onnx",
decoder="./sherpa-onnx-nemo-canary-1b-v2-int8/decoder.int8.onnx",
tokens="./sherpa-onnx-nemo-canary-1b-v2-int8/tokens.txt",
)
Or via the sherpa-onnx-ws-* / sherpa-onnx-offline CLI binaries with:
--canary-encoder=./sherpa-onnx-nemo-canary-1b-v2-int8/encoder.int8.onnx
--canary-decoder=./sherpa-onnx-nemo-canary-1b-v2-int8/decoder.int8.onnx
--tokens=./sherpa-onnx-nemo-canary-1b-v2-int8/tokens.txt
--canary-src-lang=en
--canary-tgt-lang=en
Requires sherpa-onnx >= the release that added Canary C++ runtime support (PR #2352, merged 2025-07-07) -- any sherpa-onnx release since then supports this model.
This is a community-produced ONNX conversion, uploaded to the maintainer's
own HF account. It is not published, reviewed, or endorsed by NVIDIA or
k2-fsa. k2-fsa's official asr-models release tag does not currently
include a canary-1b-v2 package (see
k2-fsa/sherpa-onnx#1767).