gigaam-multilingual-ctc-coreml

GigaAM Multilingual CTC — Core ML (fp16)

fp16 Core ML (mlprogram) export of the multilingual_ctc variant of ai-sage/GigaAM-Multilingual — a 220M-parameter Conformer encoder with a charwise CTC head for Russian, English, Kazakh, Kyrgyz, and Uzbek speech recognition. Runs fully on-device on Apple Silicon, no Python at inference time.

Unlike the RNNT family, this is a single package: one predict() per 30 s window returns CTC log-probabilities; greedy decoding is a few lines on the host. The mel front-end stays outside Core ML. The model emits plain lowercase text without punctuation (charwise vocabulary of 70 characters + blank).

Used on-device in Voicely, a local dictation, call recording, and transcription app for macOS.

Files

I/O contract

Input: features float32 [1, 64, 2999] (log-mel, 30 s window, zero-padded right), length int32 [1] (true mel frames). Output: log_probs float32 [1, 750, 71]. blank_id = 70; the 70 non-blank ids index tokens.json.

The valid decode length must be computed by the caller with integer math (fp16 cannot represent 2999 exactly inside the graph):

enc_len = (true_mel_frames - 1) // 4 + 1

verified against the PyTorch encoder's own output length across boundary values.

Mel front-end (must match exactly): 16 kHz mono, n_fft = 320, win_length = 320, hop_length = 160, center = False, 64 mel bins, HTK mel scale, no filterbank norm, then log(clamp(x, 1e-9, 1e9)). No mean/variance normalization. Frames for n samples = (n - 320) // 160 + 1.

Conversion fidelity

Token-exact against the PyTorch fp32 reference (gigaam package) on Russian and English samples decoded over the same padded 30 s window, at CPU_AND_GPU compute units; mean |Δlog_probs| ≈ 0.006–0.008.

Conversion notes

torch 2.13.0 → one warm-up forward (the rotary positional-encoding cache is built lazily) → torch.jit.trace(strict=False) → coremltools 9.0 (mlprogram, FLOAT16, macOS15). Source checkpoint: multilingual_ctc.ckpt from the official Sber CDN (MD5 5379d887c53ccd9cb95981e2a1832720, as pinned by the gigaam package). The three-package RNNT conversion recipe by smkrv documented the rotary warm-up and fp16 length pitfalls this export follows.

License and attribution

MIT, inherited from ai-sage/GigaAM-Multilingual (© GigaChat Team / SaluteDevices). Paper: GigaAM Multilingual: Foundation Model for Underrepresented Languages (Interspeech 2026). Reference implementation: salute-developers/GigaAM.