zurich-romansh-nllb-1.3b-ct2

<p align="center"> <img src="logos.png" alt="Logo of University of Zurich and Lia Rumantscha" style="max-width:400px" /> </p>

Romansh NLLB-200 1.3B CT2 Model Card

Romansh NLLB is a multilingual NMT model finetuned from NLLB-200-Distilled 1.3 model (https://huggingface.co/facebook/nllb-200-distilled-1.3B). It can translate between German and six Romansh varieties, which makes it the first model to generate fluent translations in those individual Romansh varieties.

The data used for the finetuning are monolingual data in all Romansh varieties, which we back-translated into German using Gemini 2.5 Flash, as well as a smaller amount of natively parallel data. A more extensive overview over the data used can be found further below.

The model has been extended by six language tokens, representing each of the Romansh varieties. The exact usage of the model is described below.

How to use

This model shows how to translate a sentence from German into Romansh,

Before using the model, make sure that ctranslate2 is installed. This library allows for efficient inference. This model is already converted to work with ctranslate2.

Installation:

pip install ctranslate2

Further information on ctranslate2 can be found under this link: https://github.com/OpenNMT/CTranslate2

---

Follow the below steps to translate a sentence.

1. Load model and tokenizer from repo.

import ctranslate2
from huggingface_hub import snapshot_download
from transformers import NllbTokenizer

# Download the model from the HF hub. The model is already converted to CT2 format
model_path = snapshot_download(repo_id="ZurichNLP/romansh-nllb-1.3b-ct2")

translator = ctranslate2.Translator(model_path, device="cpu")
tokenizer = NllbTokenizer.from_pretrained("ZurichNLP/romansh-nllb-1.3b-ct2")

1. Prepare model inputs.

source_lang_code = "deu_Latn" # German
target_lang_code = "rm-puter" # Puter

source_sentence = "Gestern hat es geschneit."

tokenizer.set_src_lang_special_tokens(source_lang_code)

source_token_ids = tokenizer.encode(source_sentence, add_special_tokens=True)
source_tokens = tokenizer.convert_ids_to_tokens(source_token_ids)

1. Run the model as below.

results = translator.translate_batch(
    [source_tokens],
    target_prefix=[[target_lang_code]],
    beam_size=4,
)

1. Postprocess translation as described below.

result = results[0]
target_tokens = result.hypotheses[0]
translation = tokenizer.decode(tokenizer.convert_tokens_to_ids(target_tokens), skip_special_tokens=True)
translation
# Her ho que navieu.

The overview over the six varieties and their respective language codes can be found in the below table.

Training Data

Parallel data

Monolingual data

Evaluation Results

German to Romansh BLEU:

Romansh to German BLEU:

Romansh to German COMET:

Acknowledgements

We thank RTR and Fundaziun Patrimoni Cultural RTR for their support. We are grateful to Zachary Hopton, Diana Merkle, Anna Rutkiewicz and Sudehsna Sivakumar for help with data curation, Uniun dals Grischs for contributing dictionary data for Puter and Vallader, and Giuanna Caviezel, Not Soliva and their seminar participants for helpful feedback. We also acknowledge the contribution of the native speakers who participated in the human evaluation study. For this publication, use was made of media data made available via Swissdox@LiRI by the Linguistic Research Infrastructure of the University of Zurich (see (https://www.liri.uzh.ch/en/services/swissdox.html) for more information).

License

CC-BY-NC-4.0