gigaam-v3-mlx

GigaAM v3 · MLX Runtime Bundle

Offline Russian speech recognition for Apple devices.
Native MLX inference on iPhone, iPad, and Mac — no Python, no cloud, no server required at runtime.

Converted from ai-sage/GigaAM-v3 (e2e_rnnt revision).
No additional training or fine-tuning. The native runtime produces identical decoded text to the Python reference on validated test inputs.

---

Platforms

---

Quality

Evaluated on Apple M1 Max, release build.
Normalization: lowercase, punctuation removed.

Spontaneous Speech is a challenging scenario: conversational style, filler words, non-standard vocabulary. WER on read speech is expected to be lower — and the 6.76% result on Scripted Speech confirms this.

---

Performance · Apple M1 Max

Short audio (~6 s)

The native runtime is approximately 4.2× faster than the Python reference on the same hardware.

Long-form (911 s · 15 min 11 s)

<details> <summary>Stage breakdown (long-form)</summary>

The current main bottleneck is the RNNT joint network during greedy decoding.

</details>

Benchmark · 276 clips (CV Spontaneous Speech 3.0)

Cold — each clip starts a new process including model load (~1.4 s overhead).
Warm — model loaded once, all clips processed sequentially in a single process.

Memory

---

Runtime Pipeline

Audio file / PCM samples
  → native audio loader
  → 16 kHz mono Float32 PCM
  → mel spectrogram frontend
  → Conformer encoder
  → RNNT greedy decoder
  → SentencePiece tokenizer
  → text

All preprocessing assets (hann_window, mel_filterbank) are bundled so native runtimes can reproduce the original pipeline exactly without any Python audio libraries.

---

Requirements

This bundle is designed for native Apple platform development.
It is not a Python package — there is nothing to pip install.

Native runtime (target)

Swift SDK and CLI: kruatech/gigaam-v3-mlx

To use this bundle in a native app, add the MLX Swift package to your Xcode project:

https://github.com/ml-explore/mlx-swift

---

Repository Files

README.md
.gitattributes
manifest.json
checksums.sha256
weights.fp16.safetensors
tokenizer.model
tokenizer_vocab.json
hann_window.f32.bin
mel_filterbank_mel_freq.f32.bin

---

Architecture

---

Frontend Configuration

Effective feature hop: 10 ms before encoder subsampling.
Encoder subsampling factor: 4 → one encoder frame ≈ 40 ms of audio.

---

Long-Form Transcription

Long audio is processed in chunks to keep memory usage bounded and inference latency predictable.

Recommended settings:

chunk_seconds: 20
overlap_seconds: 0–2
sample_rate: 16000
channels: mono

Future runtimes may add VAD segmentation, overlap-aware merging, and timestamp-aware chunking for improved accuracy on long-form content.

---

Validation

The conversion was validated against the original PyTorch/Hugging Face model using tensor-level golden references at each stage of the pipeline.

Validated components:

• Audio frontend · mel spectrogram
• Pre-encoder
• Conformer feed-forward blocks
• Rotary self-attention
• Conformer convolution block
• Full Conformer layer · encoder stack
• RNNT predictor · RNNT joint network
• RNNT greedy decoding
• SentencePiece tokenizer
• Full WAV-to-text pipeline (end-to-end)

Selected numerical results:

Mel frontend
  feature_shape      [64, 99]
  max_abs_diff       0.0004234314
  mean_abs_diff      2.8040542e-05

Encoder stack
  stack_max_abs_diff 2.5629997e-06
  stack_mean_abs_diff 3.8420205e-07

Full encoder
  output_shape       [1, 768, 25]
  max_abs_diff       2.682209e-06
  mean_abs_diff      4.0401252e-07

End-to-end: the native runtime and the Python reference produce identical decoded text for the same input audio.

---

Limitations

• Optimized for native Apple MLX runtimes; not intended for server or Python-based inference.
• Long audio should be processed in chunks by the host runtime.
• Word-level timestamps are not included in the bundle.
• iPadOS not yet verified on device.
• This repository contains model assets only — no application code, no Swift SDK source.

---

License

MIT — follows the license of the original ai-sage/GigaAM-v3 model.

---

Citation & Attribution

If you use this bundle, please also cite the original GigaAM model:

@misc{gigaam-v3,
  author       = {ai-sage},
  title        = {GigaAM-v3},
  year         = {2024},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/ai-sage/GigaAM-v3}}
}