nos-mt-es-arn-onnx

nos-mt-es-arn-onnx

ONNX build of proxectonos/es-arn, translating Spanish (es) to Aranese (Occitan) (oc-aranes).

The original is published only as a CTranslate2 binary, which runs only inside CTranslate2. This repository holds a HuggingFace PegasusForConditionalGeneration checkpoint and an ONNX export that were reconstructed from that binary. Nothing was retrained. The weights are the original weights.

Credit for the model belongs to Proxecto Nós (Universidade de Santiago de Compostela). Licence mit, the same as the source.

Contents

Use

This model has no sentencepiece tokenizer. Its published pipeline is Moses tokenization, then subword-nmt BPE with the @@ continuation marker. Install the two helpers and use the tokenizer shipped here:

pip install optimum[onnxruntime] sacremoses subword-nmt

from huggingface_hub import hf_hub_download
from optimum.onnxruntime import ORTModelForSeq2SeqLM
import importlib.util, sys

path = hf_hub_download("TigreGotico/nos-mt-es-arn-onnx", "nos_tokenizer.py")
spec = importlib.util.spec_from_file_location("nos_tokenizer", path)
mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)

tok = mod.NosTokenizer.from_pretrained("TigreGotico/nos-mt-es-arn-onnx", src_lang="es", tgt_lang="oc")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/nos-mt-es-arn-onnx", use_cache=True, use_merged=False)

ids = tok('El gato duerme en el sofá.', return_tensors="pt")
out = model.generate(**ids, num_beams=4, max_new_tokens=256)
print(tok.decode(out[0]))
# Eth gat dormís ath sofà.

For the int8 build add subfolder="int8".

Preprocessing

Read this before you replace the tokenizer.

• The source text gets no end-of-sentence token. The CT2 binary ships add_source_eos: false, so the encoder never saw </s>.
• Source and target have separate vocabularies. Pegasus has one, so the single table is [target vocabulary | source vocabulary] and every encoder input id is offset by 19992 (the target vocabulary size). The source half is suppressed at decode time with final_logits_bias = -1e9, so the decoder can never emit a source-side id.
• Words must be Moses-tokenized and BPE-applied first. The output is joined, @@ is removed, and the result is Moses-detokenized.
• Decoding starts from <s>.
• Both vocabularies are frequency-filtered, so rare words come back as <unk>. CTranslate2 prints <unk> as well. The upstream translate.py hides it with replace_unknowns=True, which copies the aligned source word; that needs attention alignments, which generate does not expose. nos_tokenizer.py therefore keeps <unk> in the output, so what you read is what the model produced.
• Feed one sentence at a time. The model has no document context and no language tag.

Parity with the original

15 source sentences, greedy and beam 4, exact string match against ctranslate2.Translator running the source model.bin:

Any remaining string difference is a beam-search tie, not a weight error.

Sample output

How the reconstruction works

A CTranslate2 model.bin is a flat self-describing binary: binary_version, the spec name and revision, then one record per variable (name, rank, dimensions, dtype code, byte count, raw bytes), then a table of aliases for tied weights. ct2_reader.py reads it. ct2_to_pegasus_dual.py recovers the architecture from the spec scalars and maps every variable onto a HuggingFace parameter.

This model reports:

{
  "encoder_layers": 12,
  "decoder_layers": 12,
  "source_vocab_size": 20000,
  "target_vocab_size": 19992,
  "d_model": 512,
  "heads": 16,
  "ffn_dim": 2048,
  "pre_norm": true,
  "activation": "relu",
  "layernorm_embedding": false,
  "relative_position": false,
  "scale_embeddings": true,
  "output_bias": true,
  "stored_positions": true,
  "attention_bias": false,
  "ct2_spec": "TransformerSpec rev 7, binary_version 6",
  "source_eos": false,
  "source_bos": false,
  "decoder_start_token": "<s>"
}

Why Pegasus

1. Pre-norm blocks with a final encoder and decoder layer norm, and no layernorm_embedding. That rules out BART, mBART and PLBart, whose layernorm_embedding cannot be neutralised — a LayerNorm with weight 1 and bias 0 still normalises. It also rules out Marian, which is post-norm.
2. An output bias (decoder/projection/bias). Neither Marian nor M2M100 has one. Pegasus does, as final_logits_bias.

Traps

• Pegasus refuses to save its position table. embed_positions.weight is in _keys_to_ignore_on_save and is rebuilt on load with 10000^(2i/dim). OpenNMT-py interleaves sin and cos instead. This binary does store the real table, so the converter writes it in, clears _keys_to_ignore_on_save, and reloads the checkpoint to assert the table survived. Skipping this produces a model that runs and translates plausibly but wrongly.
• CTranslate2 fuses self-attention Q, K and V into one linear_0 of shape (3d, d) in that order. Cross-attention splits differently: linear_0 is Q alone, linear_1 is [K; V] fused, linear_2 is the output projection.
• These models were trained with add_qkvbias=False, so the attention and feed-forward projections carry no bias. Zeros are written where HuggingFace insists on one.
• Weights are stored (out, in), the layout torch.nn.Linear uses, so nothing is transposed. gamma and beta are the layer-norm weight and bias.
• int8 quantization is restricted to MatMul with /lm_head/MatMul excluded. Quantizing every operator destroys a 512-dimension NMT decoder.

Attribution

Model and training data: Proxecto Nós, licence mit. Source repository: proxectonos/es-arn. The model was built for the paper Training and fine-tuning NMT models for low-resource languages using Apertium-based synthetic corpora (Sant et al., 2023), within the Nós Project funded by the Ministerio para la Transformación Digital y de la Función Pública and the EU NextGenerationEU programme (ILENIA, 2022/TL22/00215336).

This repository only changes the file format.