amalia-fala-aberto-onnx

AMALIA-FALA-ABERTO — ONNX (speech-llm)

ONNX export of amalia-llm/AMALIA-FALA-ABERTO for onnx-asr with the speech-llm model type (it needs the speech-llm model family patches, see TigreGotico/onnx-asr PR #3).

A SLAM-ASR model for European Portuguese, made by INESC-ID: the speech encoder of facebook/seamless-m4t-v2-large, a concat projector, and the 9B AMALIA language model. It was trained on the open FalAR and CAMOES corpora.

Three graphs: encoder.onnx (feature frames -> projected audio embeddings), embed_tokens.onnx, and a merged prefill and decode decoder.onnx with a 42-layer KV cache. The prompt token ids are in config.json, so no tokenizer is needed at run time. The audio embeddings come before the prompt, which is the SLAM-ASR layout. fp32 output matches the native PyTorch checkpoint character for character on two FLEURS Portuguese clips.

Hardware requirements — read this first

This is a 9B model in fp32. The graphs are about 36 GB on disk, and loading them needs about 36 GB of memory. On a 12-core CPU it transcribes 11.8 s of audio in 54 s (RTFx 0.22), so a GPU execution provider is needed for anything interactive. Plan for a GPU with 40 GB or more, or expect very slow CPU inference.

An int8 build of every graph is included. The int8 set is 9.7 GB instead of 36 GB, it loads into about 10 GB of memory, and it transcribes the same clip in 28 s (RTFx 0.43). The weights are per-tensor symmetric int8. int8 does change the transcript, so use fp32 when accuracy matters more than speed.

Shared graphs with AMALIA-SFT-FALA

SLAM-ASR training freezes the language model. Every llm.* tensor of this checkpoint is bit identical to amalia-llm/AMALIA-SFT-FALA, so embed_tokens.onnx, decoder.onnx, their int8 builds and vocab.json are the same files as in OpenVoiceOS/amalia-sft-fala-onnx. Only encoder.onnx and encoder_int8.onnx differ: this model hears through a Seamless conformer encoder, the other one through a Whisper encoder.

Audio length

encoder.onnx takes a variable number of feature frames, so short clips cost less than long ones. Full attention over the whole clip is used. This matches the source model for audio up to 400 s; past that the source model switches to chunked attention and this graph does not.

Usage

import onnx_asr
model = onnx_asr.load_model("speech-llm", "path/to/this/repo")
print(model.recognize("audio_16khz.wav"))

# or the int8 build, which is two times faster and needs a quarter of the memory
model = onnx_asr.load_model("speech-llm", "path/to/this/repo", quantization="int8")

Files

Source components

• Checkpoint: amalia-llm/AMALIA-FALA-ABERTO by amalia-llm / INESC-ID
• Language model: amalia-llm/AMALIA-9B-1225-SFT
• Speech encoder: facebook/seamless-m4t-v2-large
• Training data: inesc-id/FalAR, inesc-id/camoes_asr

License: MIT (inherited from the source checkpoint).