m2m100-418M-onnx

m2m100-418M-onnx

ONNX export of facebook/m2m100_418M, a many-to-many multilingual translation model. It translates directly between any pair of its 100 languages. No English pivot is necessary.

Two precisions are in this repository:

• fp32 — in the repository root
• int8 — in the int8/ folder (dynamic quantization of the weights)

Export command

optimum-cli export onnx \
  --model facebook/m2m100_418M \
  --task text2text-generation-with-past \
  --no-post-process \
  m2m100_418M_onnx

--no-post-process keeps the decoder graphs separate. It prevents a protobuf EncodeError during the decoder merge step.

The int8 folder was made with onnxruntime.quantization.quantize_dynamic (QuantType.QInt8, EnableSubgraph=True).

File layout

Both folders hold the full SentencePiece tokenizer (sentencepiece.bpe.model, vocab.json, tokenizer_config.json, special_tokens_map.json, added_tokens.json).

How to select the target language

M2M100 is not a prefix-token model. Do these two steps:

1. Set the source language on the tokenizer: tokenizer.src_lang = "en".
2. Give the target language to generate() as forced_bos_token_id=tokenizer.get_lang_id("pt").

If you forget forced_bos_token_id, the model gives unreliable output. Language codes are plain ISO-639-1 or ISO-639-3 strings such as en, pt, gl, ca, ar, ast.

M2M100 does not include Basque (eu). tokenizer.lang_code_to_id holds the full list of the 100 supported codes.

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

repo = "TigreGotico/m2m100-418M-onnx"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = ORTModelForSeq2SeqLM.from_pretrained(repo)          # fp32
# model = ORTModelForSeq2SeqLM.from_pretrained(repo, subfolder="int8")  # int8

tokenizer.src_lang = "en"
inputs = tokenizer("The library opens at nine in the morning.", return_tensors="pt")
ids = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.get_lang_id("gl"),
    num_beams=4,
    max_new_tokens=64,
)
print(tokenizer.batch_decode(ids, skip_special_tokens=True)[0])
# A biblioteca abre ás nove da mañá.

Parity with the PyTorch original

12 sentences over 6 language directions (en→pt, en→gl, en→ca, en→ast, en→ar, pt→en), num_beams=4, max_new_tokens=64, greedy string comparison against M2M100ForConditionalGeneration.

The one int8 difference is a word choice in en→ca. It is still a correct translation.

The parity set uses Asturian (ast) in place of Basque, because M2M100 has no Basque support.

Licence

MIT, the same as the original model.