translate-eus-cat-onnx

translate-eus-cat-onnx

ONNX build of softcatala/translate-eus-cat, translating Basque (eu) to Catalan (ca).

The original is published only as a CTranslate2 binary, which runs only inside CTranslate2. This repository holds a HuggingFace PegasusForConditionalGeneration checkpoint and an ONNX export that were reconstructed from that binary. Nothing was retrained. The weights are the original weights.

Credit for the model belongs to Softcatalà. Licence mit, the same as the source.

Contents

Use

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("TigreGotico/translate-eus-cat-onnx")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/translate-eus-cat-onnx", use_cache=True, use_merged=False)

ids = tok('Katua sofan lo dago.', return_tensors="pt")
out = model.generate(**ids, num_beams=4, max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))
# El gat dorm al sofà.

For the int8 build add subfolder="int8".

Preprocessing

Read this before you replace the tokenizer.

• The source text gets no end-of-sentence token. The CT2 binary reports with_source_eos = 0, so the encoder never saw </s> during training. MarianTokenizer, PegasusTokenizer and T5Tokenizer all append </s>, which puts the model off its training distribution. The tokenizer shipped here appends nothing.
• Token ids follow the CT2 shared_vocabulary order, which is not the sentencepiece order: CT2 puts <blank> (the pad token) at id 0 and moves <unk> to the end of the table.
• Decoding starts from <s>, not from </s>.
• Feed one sentence at a time, as the source model does. The model has no document context and no language tag.

Parity with the original

15 source sentences, greedy and beam 4, exact string match against ctranslate2.Translator running the source model.bin:

Token-level agreement on the CT2 hypothesis is about 0.005 maximum absolute log-probability difference, which is kernel numeric noise. Any remaining string difference is a beam-search tie, not a weight error.

How the reconstruction works

A CTranslate2 model.bin is a flat self-describing binary: binary_version, the spec name and revision, then one record per variable (name, rank, dimensions, dtype code, byte count, raw bytes), then a table of aliases for tied weights. ct2_reader.py reads it and dequantizes int8 variables with their *_scale companion. ct2_to_pegasus.py recovers the architecture from the spec scalars and maps every variable onto a HuggingFace parameter.

This model reports:

{
  "encoder_layers": 6,
  "decoder_layers": 6,
  "vocab_size": 50001,
  "d_model": 512,
  "heads": 8,
  "ffn_dim": 2048,
  "pre_norm": true,
  "activation": "relu",
  "layernorm_embedding": false,
  "relative_position": false,
  "scale_embeddings": true,
  "output_bias": true,
  "stored_positions": false,
  "source_bos": false,
  "source_eos": false,
  "target_bos": true,
  "decoder_start_token": "<s>",
  "ct2_spec": "TransformerSpec rev 7, binary_version 6"
}

Why Pegasus

Three properties fix the target class:

1. Pre-norm blocks with a final encoder and decoder layer norm, and no layernorm_embedding. That rules out BART, mBART and PLBart, whose layernorm_embedding cannot be neutralised — a LayerNorm with weight 1 and bias 0 still normalises. It also rules out Marian, which is post-norm.
2. An output bias (decoder/projection/bias). Neither Marian nor M2M100 has one. Pegasus does, as final_logits_bias.
3. A single shared vocabulary, aliased to the encoder embeddings and to the output projection. Pegasus ties all three.

Traps

• The sinusoidal position table is not stored in model.bin. CTranslate2 builds it with the OpenNMT-tf formula, spacing log(10000)/(dim/2 - 1) and layout [sin all | cos all]. Pegasus rebuilds it on load with 10000^(2i/dim), which is a different table. The converter writes the CT2 table into embed_positions.weight, clears _keys_to_ignore_on_save so it is actually persisted, and reloads the checkpoint to assert the table survived. Skipping this produces a model that runs and translates plausibly but wrongly.
• Position 0 is the first real token. OpenNMT-tf has no fairseq padding offset. Pegasus indexes the same way, so no realignment is needed here — but M2M100 would have needed one.
• CTranslate2 fuses self-attention Q, K and V into one linear_0 of shape (3d, d) in that order. Cross-attention splits differently: linear_0 is Q alone, linear_1 is [K; V] fused, linear_2 is the output projection.
• Weights are stored (out, in), the layout torch.nn.Linear uses, so nothing is transposed. gamma and beta are the layer-norm weight and bias.
• int8 quantization is restricted to MatMul with /lm_head/MatMul excluded. Quantizing every operator destroys a 512-dimension NMT decoder.

Attribution

Model and training data: Softcatalà, licence mit. Source repository: softcatala/translate-eus-cat. This repository only changes the file format.