gigachat-audio-mlx

GigaChat Audio MLX — BF16

English · Документация на русском

Native BF16 MLX conversion of ai-sage/GigaChat3.1-Audio-10B-A1.8B for Apple Silicon. It accepts audio plus a text instruction and produces text: audio QA, transcription, translation, emotion analysis, temporal grounding, and timestamped summaries.

This is an independent port, not an official release by the upstream authors.

GitHub · PyPI · All models · Full results · Upstream model · Paper

Models

• gigachat-audio-mlx — BF16 · best measured quality ← this model
• gigachat-audio-mlx-q8-bf16 — Q8/BF16 · smaller and faster

<div>_{* Time to process a 5 min 17 sec audio file on Apple M4 Pro. Lower is better.}</div>

Choose Q8/BF16 when size, memory, and speed matter; choose BF16 for the best measured answer quality. Both variants use the same gigachat-audio-mlx runtime.

The original model and paper are by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin. The upstream model card declares MIT. See the runtime license and third-party notices.

What do I need to run it?

<div>_{* Time to process a 5 min 17 sec audio file on Apple M4 Pro. Lower is better.}</div>

The free-disk figure is a rounded practical reserve; the CLI dry run reports the exact value. Memory is measured use, not a minimum-memory guarantee. This release is verified only on a 48 GB Mac; 16/24/32 GB support is not yet claimed.

Why MLX matters on this Mac

Same Apple M4 Pro / 48 GB. Original is a stopped memory-safety smoke, not a completed speed benchmark; MLX values are medians of five clean runs.

The pinned upstream Transformers path is demonstrated for CUDA. On MPS it exceeded this Mac's safe memory budget and nearly filled system swap during the short smoke. The 5-minute Original run was not started, so no speedup ratio is claimed. MLX is what makes the model usable on the tested Mac.

Quick start

brew install uv
uv tool install gigachat-audio-mlx
gigachat-audio-mlx doctor

gigachat-audio-mlx download --variant bf16 --dry-run

gigachat-audio-mlx download --variant bf16 --yes

The dry run reports the exact download size, cache hits, free disk space, and destination before any network transfer. The runtime never executes remote model code.

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768

Exact task examples

The text instruction is part of every input. It must not be omitted when interpreting an answer.

Russian speech transcription

Prompt:

Transcribe the Russian speech exactly. Return only the transcript.

Output:

Всё во Вселенной состоит из материи. Вся материя состоит из мельчайших частиц, называемых атомами.

Temporal grounding

Prompt:

Укажи таймкод начала и конца речи в аудио.

Output:

Таймкоды начала и конца речи в аудио:
- Начало: 00:00
- Конец: 00:20

Artifact

The model repository contains no executable Python. model_manifest.json pins the source lineage and SHA-256 of every data file. Loading fails on a missing, changed, or unexpected artifact.

Results available today

The BF16 port is the accepted native MLX baseline. In the frozen 50-item parity gallery, its decoded text exactly matched the original PyTorch BF16 output on 41/50 inputs. This is an implementation-parity diagnostic, not an aggregate quality score.

What changes with Q8/BF16

Lower is better in every row. Both models used the same MLX workloads.

The 5-minute workload contains 317.27 seconds of audio. Resource values are the median of five clean processes on the verified M4 Pro / 48 GB machine.

Q8/BF16 is the smaller, faster option; BF16 provides the best measured answer quality. In the blinded review, Q8/BF16 had factual errors in 3 more cases.

From Original to MLX

The post-publication Original PyTorch/MPS resource check did not complete even the 9.84-second smoke on this 48 GB Mac. After 277 seconds its process footprint had reached at least 51.92 GiB and system swap was nearly full, so it was stopped before the 5-minute run. Original therefore remains N/A in the direct Mac RAM and speed table; no latency is inferred from a failed, swap-dominated run.

Results reported by the Original authors

These values come from the upstream model card; we did not rerun them.

Full public release evidence separates ASR, translation, emotion, temporal localization, timed descriptions/summaries, factual review, memory, and speed. It does not collapse these tasks into one number. See the full results table.

Limitations

• Verified only on Apple M4 Pro with 48 GB unified memory and macOS 15.7.7.
• No claim is made for 16/24/32 GB machines or every M1–M5 generation.
• No 90-minute-audio support claim is made in this release.
• Generated facts, emotion labels, summaries, and timestamps can be wrong and must be checked against the source audio.
• Long summaries may require a larger output-token budget, which also changes the memory estimate.
• No diarization, streaming microphone mode, speech synthesis, training, or fine-tuning is included.
• Do not use the output as the sole basis for medical, legal, employment, safety, or surveillance decisions.

Users are responsible for consent, privacy, copyright, and dataset terms for audio they process.

License and attribution

• Upstream model: https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B
• Upstream revision: bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb
• Paper: GigaChat Audio: Time-aware Large Audio Language Model
• Original authors: Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
• Upstream license declaration: MIT in the upstream model card
• Runtime and MIT license: https://github.com/ai-babai/gigachat-audio-mlx
• Model collection: https://huggingface.co/collections/ai-babai/gigachat-audio-mlx-6a5cab53ded991903b664437

Please cite both the original paper above and this project through CITATION.cff. See the upstream model card and THIRD_PARTY_NOTICES.md for full attribution and provenance.