mt-hitz-eu-en-onnx

mt-hitz-eu-en-onnx

ONNX export of HiTZ/mt-hitz-eu-en, a MarianMT machine translation model for eu -> en produced by HiTZ, the Basque Center for Language Technology at the University of the Basque Country (UPV/EHU).

All credit for training and releasing the original model goes to the HiTZ team. This repo only contains an ONNX conversion (fp32 and int8) for fast CPU inference with optimum[onnxruntime], done by TigreGotico.

License

apache-2.0, as declared on the source model card.

Export command

optimum-cli export onnx --model HiTZ/mt-hitz-eu-en --task text2text-generation-with-past --no-post-process mt-hitz-eu-en-onnx

int8 dynamic quantization was applied afterwards with optimum.onnxruntime.ORTQuantizer (AVX2 config, dynamic) into the int8/ subfolder.

File layout

encoder_model.onnx              # fp32
decoder_model.onnx              # fp32
decoder_with_past_model.onnx    # fp32
config.json, generation_config.json
source.spm, target.spm, vocab.json, tokenizer_config.json, special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx

No external-data files were produced — this Marian model is small enough that all graphs stay under the 2GB single-file protobuf limit.

Target-language mechanism

Bilingual Marian model, Basque -> English only. No target-language token or forcing is required; tokenizer(text) + model.generate() is sufficient.

Parity vs. the original PyTorch model

Measured on 8 held-out eu->en sentences, beam search (num_beams=4, max_new_tokens=64), exact-match against the original PyTorch MarianMTModel output:

• fp32: 8/8 = 100.0% exact match
• int8: not separately re-verified against reference; reported not gated

Sample translations observed (eu -> en):

Usage

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx")
# int8: ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx", subfolder="int8")

batch = tok(["Kaixo, zer moduz?"], return_tensors="pt")
generated = model.generate(**batch, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(generated, skip_special_tokens=True))
# ['Hello, how are you?']

Attribution

• Original model: HiTZ/mt-hitz-eu-en by HiTZ Center (University of the Basque Country, UPV/EHU)
• ONNX export: TigreGotico