wmt23-cometkiwi-da-xl-eole-fp16

WMT23 COMET-KIWI DA XL FP16 (EOLE)

This is Unbabel/wmt23-cometkiwi-da-xl converted to EOLE format.

Model weights were dtype-cast to fp16 during EOLE conversion. This repository contains a format and dtype conversion for use with EOLE's native transformer_encoder_scorer implementation.

This is not an upstream unbabel-comet checkpoint layout. Use it with EOLE.

Model Details

Usage

Requires an EOLE version with native COMET scorer support.

Validation Metric

valid_metrics: ["EOLE-COMET-KIWI"]
comet_model: eole-nlp/wmt23-cometkiwi-da-xl-eole-fp16
comet_batch_size: 32

Direct Scoring

COMET-KIWI is reference-free. Use --src for source sentences and --tgt for machine translations.

eole predict \
  --model_path eole-nlp/wmt23-cometkiwi-da-xl-eole-fp16 \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/scores.txt \
  --with_score

By default, direct scoring writes one segment score per input line. To emit a single aggregate system score, add --score_level system:

eole predict \
  --model_path eole-nlp/wmt23-cometkiwi-da-xl-eole-fp16 \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --output /path/to/system-score.txt \
  --with_score \
  --score_level system

score_level: system writes one numeric line containing the arithmetic mean of the segment scores, similar to Unbabel COMET's --only_system mode but using EOLE's score-file format.

Conversion

Converted with:

eole convert COMET \
  --model Unbabel/wmt23-cometkiwi-da-xl \
  --dtype fp16 \
  --output wmt23-cometkiwi-da-xl-eole-fp16

Parity

The fp32 EOLE conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:

The fp16 artifacts are intended to provide smaller EOLE model files and fp16 runtime behavior. Run the EOLE COMET parity harness against these fp16 repositories if exact fp16 deltas are required.

Original Model and Attribution

This model is a conversion of Unbabel/wmt23-cometkiwi-da-xl. The model weights were converted to EOLE format and dtype-cast to fp16.

This converted repository follows the upstream model license: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC-BY-NC-SA-4.0). Use is subject to the upstream non-commercial, attribution, and share-alike terms. Please refer to the original model card and license for training data, intended use, limitations, and citation details.

Unbabel COMET: https://github.com/Unbabel/COMET