omniASR-CTC-7B-ONNX

omniASR-CTC-7B-ONNX

A dynamic-shape ONNX export of Meta's omniASR_CTC_7B for inference with fast-omniasr.

Usage

Install the ONNX Runtime and Hugging Face Hub extras:

python -m pip install "fast-omniasr[onnx,hub]"

from fast_omniasr import OmniASR

model = OmniASR.from_pretrained("EmreAkgul/omniASR-CTC-7B-ONNX", backend="onnx")
result = model.transcribe("speech.wav")
print(result.text)

The input must be mono audio. fast-omniasr resamples supported audio files to 16 kHz and normalizes them before inference. Larger model variants require substantially more memory.

Model and conversion

• Upstream model: Meta Omnilingual ASR, omniASR_CTC_7B
• Task: multilingual automatic speech recognition with a CTC output head
• Conversion: official fairseq2 checkpoint to ONNX with torch.onnx.export, opset 18, dynamo=False; no retraining or intentional weight modification
• Tokenizer: upstream omniASR_tokenizer.model, included unmodified as tokenizer.model
• Input: audio, float32, shape [1, samples]
• Output: logits, float32, shape [1, frames, 10288]

config.json records SHA-256 hashes and sizes for every runtime asset. OmniASR.from_pretrained() downloads and verifies that manifest. Large exports store tensor data in companion files referenced by model.onnx; all such files are required and are downloaded automatically.

Intended use and limitations

This repository is a format conversion intended for inference with fast-omniasr. It does not change the upstream model's capabilities, language coverage, accuracy, or limitations. Consult the upstream repository for model details, training data, evaluation results, and responsible-use considerations.

ONNX FP32 is the verified runtime path. TensorRT support in fast-omniasr is experimental; FP16 can change recognition results. This repository contains a CTC model and does not support the OmniASR LLM/seq2seq inference path.

Precision

from_pretrained(..., precision=...) downloads only the requested variant (plus the shared tokenizer) and needs fast-omniasr 0.3.0 or newer.

from fast_omniasr import OmniASR

model = OmniASR.from_pretrained("EmreAkgul/omniASR-CTC-7B-ONNX", device="cuda", precision="fp16")
print(model.transcribe("speech.wav").text)

fp16 and bf16 store the MatMul weights in half precision, roughly halving the download and the weight memory. Convolutions, normalization, softmax and all inputs and outputs stay float32. On CPU fp16 saves memory but is not faster; ONNX Runtime has no CPU bf16 kernels. Agreement was measured on English, Turkish and Central Kurdish clips (parity/precision_agreement.json); half precision can change individual words where fp32 itself is close to a tie, as upstream's bf16 default does.

Repository files

License and attribution

The upstream Omnilingual ASR code and model weights are licensed under Apache-2.0. This repository redistributes a format-converted version under the same license and identifies the conversion above. It is an independent distribution and is not affiliated with or endorsed by Meta.