aina-translator-gl-ca-onnx

aina-translator-gl-ca-onnx

ONNX build of projecte-aina/aina-translator-gl-ca.

The original is published only as a CTranslate2 binary. This repository holds a HuggingFace M2M100ForConditionalGeneration checkpoint and an ONNX export that were reconstructed from that binary. No retraining took place. The weights are the original weights.

Credit for the model belongs to Projecte AINA (Barcelona Supercomputing Center / Generalitat de Catalunya). Licence: apache-2.0, the same as the source.

Contents

Use

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("TigreGotico/aina-translator-gl-ca-onnx")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/aina-translator-gl-ca-onnx", use_cache=True, use_merged=False)

ids = tok("O goberno aprobou unha nova lei sobre o cambio climático.", return_tensors="pt")
out = model.generate(**ids, num_beams=4, max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))

For the int8 build add subfolder="int8".

int8 quantization is restricted to MatMul with /lm_head/MatMul excluded. Quantizing every operator destroys a 512-dimension NMT decoder.

How the reconstruction works

A CTranslate2 model.bin is a flat, self-describing binary: a version header, a spec name and revision, then one record per variable (name, rank, dimensions, dtype code, byte count, raw data), then a table of aliases for tied weights. The converter reads that file, recovers the architecture from the spec scalars, and maps every variable onto a HuggingFace parameter name.

This model reports:

{
  "encoder_layers": 24,
  "decoder_layers": 6,
  "vocab_size": 55328,
  "d_model": 1024,
  "heads": 16,
  "ffn_dim": 4096,
  "pre_norm": true,
  "activation": "relu",
  "layernorm_embedding": false,
  "relative_position": false,
  "scale_embeddings": 32.0,
  "max_positions": 1024,
  "source_eos": true,
  "source_bos": false,
  "target_bos": false,
  "tied_embeddings": true,
  "decoder_start_token": "</s>"
}

Pre-norm blocks, a final encoder and decoder layer norm, RELU feed-forwards, sinusoidal position encodings with the fairseq offset of 2, embeddings scaled by sqrt(d_model), and one shared embedding matrix aliased to the encoder embeddings and to the output projection. That is exactly M2M100ForConditionalGeneration, so it is the target architecture. Marian's HuggingFace class does not fit, because it is post-norm and uses learned positions.

Mapping notes that are not obvious:

• CTranslate2 fuses self-attention Q, K and V into one linear_0 of shape (3*d, d), in that order. Cross-attention is split differently: linear_0 is Q alone, linear_1 is [K; V] fused, linear_2 is the output projection.
• Weights are stored (out, in), the same layout torch.nn.Linear uses, so no transpose is needed.
• gamma and beta are the layer-norm weight and bias.
• The decoder/projection/weight and encoder/embeddings_0/weight aliases mean the embedding matrix is shared three ways; lm_head is tied.
• The decoder start token is </s>, not <s>. Starting from <s> degrades the first-token log probability by about 2 nats.

Parity

Every conversion is gated on agreement with the original CTranslate2 model. 20 source sentences, greedy and beam-4, exact string match of the output:

Token-level check on the same hypotheses: mean absolute log-probability difference against CTranslate2 is about 0.03, which is kernel-level numeric noise. The remaining string differences are beam-search ties, not weight errors.

Converter

ct2_to_hf.py and ct2_reader.py in this repository do the whole conversion:

python ct2_to_hf.py <ct2_model_dir> <output_hf_dir>