sensevoice-asr

SenseVoice ASR — ONNX (int8)

ONNX export and int8 quantization of SenseVoice by FunAudioLLM (Alibaba/DAMO Academy). Exported via sherpa-onnx by k2-fsa.

This repo mirrors the model for CDN stability as a dependency of the @omote/core SDK.

Model Summary

Files

Quick Start

TypeScript (@omote/core)

import { createSenseVoice } from '@omote/core';

// Zero-config — uses this HuggingFace CDN automatically
const asr = createSenseVoice();
await asr.load();

const { text, emotion, language } = await asr.transcribe(audioSamples);
console.log(`[${language}] [${emotion}] ${text}`);

Python (ONNX Runtime)

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession("model.int8.onnx", providers=["CPUExecutionProvider"])

# Input: preprocessed features (fbank → LFR → CMVN)
# See "Preprocessing" section below
feeds = {
    "x": features,           # [1, num_lfr_frames, 560] float32
    "x_length": np.array([num_lfr_frames], dtype=np.int32),
    "language": np.array([0], dtype=np.int32),   # 0=auto
    "text_norm": np.array([14], dtype=np.int32), # 14=with_itn
}

logits = session.run(["logits"], feeds)[0]
# CTC decode logits to text

Input / Output Specification

Inputs

Outputs

Preprocessing

All preprocessing is external to the ONNX model. The model expects already-preprocessed features:

1. Kaldi Fbank: Raw 16kHz audio → 80-bin log-mel spectrogram (25ms frame, 10ms shift, Hamming window, 0.97 preemphasis)
2. LFR Stacking: Stack 7 frames with stride 6 → [T_reduced, 560]
3. CMVN: Normalize using statistics stored in model metadata (neg_mean, inv_stddev — 560 floats each)

The @omote/core SDK handles all preprocessing automatically.

CMVN Statistics

Stored as comma-separated strings in the ONNX model's custom metadata:

• neg_mean: 560 float values (negative mean for normalization)
• inv_stddev: 560 float values (inverse standard deviation)

Access in Python: session.get_modelmeta().custom_metadata_map["neg_mean"]

Attribution

• Model: SenseVoiceSmall by FunAudioLLM (Alibaba/DAMO Academy)
• Paper: FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
• ONNX Export: sherpa-onnx by k2-fsa (@csukuangfj)
• Original HuggingFace: csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17

License

Model weights are licensed under the FunASR Model License v1.1 by Alibaba Group. The license permits copying, modifying, and sharing with attribution.

The ONNX export code (sherpa-onnx) is licensed under Apache 2.0.