gigaam-v3-coreml

GigaAM v3 (e2e CTC) — Core ML

Core ML conversion of Sber's GigaAM v3 v3_e2e_ctc for Apple silicon. Runs on the Neural Engine, produces Russian text with punctuation and capitalization out of the box.

Attribution

Converted from the original PyTorch checkpoint (gigaam.load_model("v3_e2e_ctc")), not from an intermediate ONNX export. Weights are redistributed under the original MIT license.

Files

The three small files are not optional. The model takes mel features, not audio, and emits token ids — without the filterbank, the window and the vocabulary the package cannot be used.

Input / output

Output names are generated by the converter and carry no meaning. Matching them by rank — the 3-D tensor is the logits — is more robust than matching by name.

Preprocessing

The feature extractor must reproduce the checkpoint's own configuration exactly:

sample rate  16000 Hz, mono
n_fft        320
win_length   320   (Hann, periodic)
hop_length   160
center       false
n_mels       64    (HTK scale, 0–8000 Hz, unnormalized)
power        2
output       log(clamp(mel, 1e-9, 1e9))
frames       floor((samples - 320) / 160) + 1

Do not compute the mel filterbank from a formula. GigaAM ships its own filterbank inside the checkpoint, it overwrites the standard torchaudio one on load, and it matches no standard recipe. gigaam-melfb.f32 and gigaam-window.f32 are those exact tables.

Audio is padded with zeros to the 25 s window; the unpadded frame count goes into feature_lengths. Longer audio has to be chunked by the caller.

Decoding

Greedy CTC: argmax per frame → collapse repeats → drop blank (id 256) → concatenate pieces, where a leading ▁ marks a word boundary and becomes a space. Token id 0 is <unk> and is skipped. The sentencepiece library is not needed at run time — the flat vocabulary is enough.

How it was converted

Environment: Python 3.13, torch==2.7.*, torchaudio==2.7.*, coremltools>=9, gigaam from its git repository. Target ct.target.macOS14, weights in fp16, format .mlpackage.

The conversion is not a one-liner, and most of the work is in getting a clean graph out of PyTorch:

1. Export forward_for_export(features, lengths) (encoder + head), not forward() — the latter expects a raw waveform and calls the preprocessor internally.
2. Do not call transcribe() before tracing. Its @inference_mode caches rotary cos/sin as inference tensors and the trace fails afterwards. Warm the model up through forward_for_export under no_grad instead.
3. Trace inside model.encoder.onnx_export_mode().
4. torch 2.13 is incompatible with coremltools 9 (aten::Int bug) — use 2.7.x.
5. torch.jit.trace fails even on 2.7 (the same integer bug in positional encoding). Use torch.export.export followed by ep.run_decompositions({}).
6. The example input must be .contiguous() (EXIR rejects a non-contiguous dim order), but do not put .contiguous() inside forward — that introduces an alias node.
7. The decomposed graph still contains 32 aten.alias nodes, which coremltools cannot lower. They are no-ops: walk the FX graph, replace_all_uses_with(node.args[0]), erase_node, then lint() and recompile().

Dynamic input length did not survive conversion, hence the fixed 25 s window.

import gigaam, torch
import coremltools as ct

model = gigaam.load_model("v3_e2e_ctc", device="cpu"); model.eval()
wav, length = model.prepare_wav("sample_25s.wav")
with torch.no_grad():
    features, feat_len = model.preprocessor(wav, length)
    features = features.contiguous()

class W(torch.nn.Module):
    def __init__(s, m): super().__init__(); s.m = m
    def forward(s, features, feature_lengths):
        return s.m.forward_for_export(features, feature_lengths.to(torch.long))

w = W(model).eval()
with model.encoder.onnx_export_mode(), torch.no_grad():
    ep = torch.export.export(w, (features, feat_len.to(torch.int32)))
    ep = ep.run_decompositions({})
    gm = ep.graph_module
    for node in list(gm.graph.nodes):
        if node.op == "call_function" and "alias" in str(node.target):
            node.replace_all_uses_with(node.args[0])
            gm.graph.erase_node(node)
    gm.graph.lint(); gm.recompile()
    mlm = ct.convert(ep, minimum_deployment_target=ct.target.macOS14)
mlm.save("gigaam_v3_e2e.mlpackage")

Usage

import CoreML

let cfg = MLModelConfiguration()
cfg.computeUnits = .all                     // let the system pick ANE/GPU/CPU
let model = try MLModel(contentsOf: compiledURL, configuration: cfg)

// features: [1, 64, 2499] float32 log-mel, zero-padded to the 25 s window
// realFrames: frame count before padding
let lens = try MLMultiArray(shape: [1], dataType: .int32)
lens[0] = NSNumber(value: Int32(realFrames))

let out = try model.prediction(from: MLDictionaryFeatureProvider(
    dictionary: ["features": features, "feature_lengths": lens]))

// 3-D output is the logits; the other one is the encoded length

A .mlpackage has to be compiled before use — either MLModel.compileModel(at:) once at install time, or xcrun coremlcompiler compile gigaam_v3_e2e.mlpackage . ahead of time.

Measurements

MacBook Air, Apple silicon, 8 GB, 25 s of audio:

Limitations

• Fixed 25 s input window; dynamic shapes did not survive the conversion. Longer audio must be chunked, and chunk boundaries need de-duplication if the chunks overlap.
• Input is mel features, not audio. The feature extractor is the caller's responsibility and must match the tables shipped here.
• fp16 weights: 1.6% of argmax tokens differ from the PyTorch reference. Decoded text was identical on the samples tested, but this is not a guarantee.
• Russian only. English words are usually transliterated into Cyrillic.
• Greedy CTC only. No beam search, no language model, no timestamps, no diarization.
• Deployment target macOS 14. Not tested on iOS.

Used by

Voica — a macOS menu-bar dictation app; this model is its offline engine.

Support

A by-product of a personal project, published because it may be useful. Issues are read, but answers may be slow or absent, and feature requests are not accepted. There is no commitment to convert future GigaAM releases.