silero_vad_ggml-base

Silero VAD & Whisper GGML Base (Local STT Bundle) 🎙️

This repository contains a curated set of models for building fully local Speech-to-Text (STT) systems. The bundle is intended for .NET 10 (C#) applications using Whisper.net and ONNX Runtime, but the model files can also be used with other compatible runtimes.

---

📦 Repository Contents

1. Silero VAD (Voice Activity Detection)

• File: silero_vad.onnx

• Format: ONNX

• Purpose: Real-time detection of human speech within an audio stream.

• Model: Silero VAD v5

• Technical Specifications:

  • 16 kHz mode: 512 new audio samples per inference step.

  • 8 kHz mode: 256 new audio samples per inference step.

  • State: float32 [2, batch, 128] — recurrent model state that must be preserved between chunks.

  • Sample Rate: int64 — 16000 or 8000.

  • Outputs:

    • output — speech probability.
    • stateN — updated recurrent state for the next inference step.

Silero VAD v5 also uses a small amount of audio context from the previous chunk. At 16 kHz, 64 previous samples are prepended to each 512-sample window before ONNX inference; at 8 kHz, 32 previous samples are prepended to each 256-sample window.

This bundle uses the 16 kHz / 512-sample path so the same normalized audio stream can be passed directly to Whisper after speech segmentation.

2. OpenAI Whisper (Base)

• File: ggml-base.bin
• Format: GGML
• Compatible with: whisper.cpp and Whisper.net
• Purpose: Multilingual speech transcription and speech translation into English.
• Architecture: Transformer (~74M parameters).
• Model Variant: Multilingual Whisper Base.

GPU acceleration is provided by the inference runtime rather than by the model file itself. For example, Whisper.net.Runtime.Vulkan provides a Vulkan-enabled whisper.cpp runtime for supported systems.

---

🛠 Technical Audio Specifications

The reference pipeline normalizes incoming audio to a common format before VAD and Whisper processing:

Using one shared 16 kHz mono stream avoids unnecessary resampling between the VAD and transcription stages.

---

🚀 Reference Pipeline

A low-latency implementation can use a concurrent producer/consumer pipeline with System.Threading.Channels:

1. Stage 1: Audio Processor Decode the incoming audio format, resample it to 16 kHz mono, normalize it to float PCM, and stream 512-sample chunks into the VAD stage.

2. Stage 2: VAD Processor Consume audio chunks, maintain the Silero recurrent state and previous-audio context, detect speech, and group active speech into complete utterances using configurable silence thresholds.

3. Stage 3: Transcription Send finalized speech segments to Whisper while the next audio segment is still being received and processed.

This allows audio decoding, voice activity detection, and Whisper inference to overlap instead of processing the entire recording sequentially.

---

💡 Quick Start Notes

1. VAD State Management Initialize the Silero recurrent state to zeros at the beginning of a new stream. After every inference call, preserve the returned stateN and pass it into the next call. The previous audio context must also be preserved between chunks.

2. Whisper GPU Acceleration GPU acceleration depends on the selected Whisper.net / whisper.cpp runtime. Whisper.net.Runtime.Vulkan can be used for Vulkan acceleration on supported Windows x64 systems. Other Whisper.net runtimes are available for CPU, CUDA, CoreML, and OpenVINO.

3. Concurrency When serving multiple STT requests, limit concurrent Whisper inference according to available CPU/GPU resources and VRAM. SemaphoreSlim is a simple option for controlling concurrent inference in .NET.

---

⚖️ License & Credits

The original models and their upstream projects are distributed under the MIT License.

• Silero VAD — developed by the Silero Team.
• Whisper — developed and released by OpenAI.
• GGML model format and runtime compatibility — provided by whisper.cpp.

This repository does not claim authorship of the original model architectures or weights. The applicable upstream copyright and MIT license notices remain in effect when redistributing the model files.