m2m100-1.2B-onnx

m2m100-1.2B-onnx

ONNX export of facebook/m2m100_1.2B, a many-to-many multilingual translation model. It translates directly between any pair of its 100 languages. No English pivot is necessary.

Two precisions are in this repository:

• fp32 — in the repository root
• int8 — in the int8/ folder (dynamic quantization of the weights)

For a smaller model, see TigreGotico/m2m100-418M-onnx.

Export command

optimum-cli export onnx \
  --model facebook/m2m100_1.2B \
  --task text2text-generation-with-past \
  --no-post-process \
  m2m100_1.2B_onnx

--no-post-process keeps the decoder graphs separate. It prevents a protobuf EncodeError on graphs that are larger than 2 GB.

The int8 folder was made with onnxruntime.quantization.quantize_dynamic (QuantType.QInt8, EnableSubgraph=True).

File layout

The fp32 graphs keep their weights in external *.onnx_data files. You must download those files together with the .onnx files. The graphs do not load without them. The int8 graphs are each below 2 GB, so they need no external data.

Both folders hold the full SentencePiece tokenizer (sentencepiece.bpe.model, vocab.json, tokenizer_config.json, special_tokens_map.json, added_tokens.json).

How to select the target language

M2M100 is not a prefix-token model. Do these two steps:

1. Set the source language on the tokenizer: tokenizer.src_lang = "en".
2. Give the target language to generate() as forced_bos_token_id=tokenizer.get_lang_id("pt").

If you forget forced_bos_token_id, the model gives unreliable output. Language codes are plain ISO-639-1 or ISO-639-3 strings such as en, pt, gl, ca, ar, ast.

M2M100 does not include Basque (eu). tokenizer.lang_code_to_id holds the full list of the 100 supported codes.

Usage

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

repo = "TigreGotico/m2m100-1.2B-onnx"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = ORTModelForSeq2SeqLM.from_pretrained(repo)          # fp32
# model = ORTModelForSeq2SeqLM.from_pretrained(repo, subfolder="int8")  # int8

tokenizer.src_lang = "en"
inputs = tokenizer("The library opens at nine in the morning.", return_tensors="pt")
ids = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.get_lang_id("gl"),
    num_beams=4,
    max_new_tokens=64,
)
print(tokenizer.batch_decode(ids, skip_special_tokens=True)[0])
# A biblioteca abre ás nove da mañá.

Parity with the PyTorch original

12 sentences over 6 language directions (en→pt, en→gl, en→ca, en→ast, en→ar, pt→en), num_beams=4, max_new_tokens=64, greedy string comparison against M2M100ForConditionalGeneration.

Each of the 4 int8 differences is still a correct translation. The differences are word choice and number formatting, for example surt en 20 minuts against surt en vint minuts. Use fp32 if you need output that is identical to PyTorch. Use int8 for a 4x smaller model with translation quality that is very close.

The parity set uses Asturian (ast) in place of Basque, because M2M100 has no Basque support.

Licence

MIT, the same as the original model.