mt0-definition-de-xl-dbnary

ltg/mt0-definition-de-xl-dbnary

This model is a version of bigscience/mt0-xl, fine-tuned on datasets of German usage examples and definitions.

It generates definitions of German words in context. Its input is the usage example and the instruction question ". Was ist die Definition von <target word>?"

Other models

Finnish

• decoder-only

Tower, axolotl24 Tower, axolotl24 + dbnary

• encoder-decoder

mT0-xl, axolotl24 mT0-xl, axolotl24 + dbnary

aya-101, axolotl24 aya-101, axolotl24 + dbnary

German

• decoder-only

Tower, dbnary

• encoder-decoder

mT0-xl, dbnary

aya-101, dbnary

Russian

• decoder-only

Tower, axolotl24 Tower, axolotl24 + dbnary

• encoder-decoder

mT0-xl, axolotl24 mT0-xl, axolotl24 + dbnary

aya-101, axolotl24 aya-101, axolotl24 + dbnary

More context

• Github repository: MultilingualDefGen
• Paper: EMNLP 2025 Findings

Uses

The model is intended for research purposes, as a source of contextualized dictionary-like lexical definitions.

The fine-tuning datasets were limited to German. Although the original model is multilingual, we did not evaluate its ability to generate definitions in other languages.

Generated definitions can contain all sorts of biases and stereotypes, stemming from the underlying language model and raw dictionary data.

Direct Use

script to run prediction

Training Details

Training Data

dbnary

Training Procedure

script to run training

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

run evaluation

Testing Data

<!-- This should link to a Dataset Card if possible. -->

axolotl24 German test set

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

BLEU, BERTScore

Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

@inproceedings{fedorova-etal-2025-explaining,
    title = "Explaining novel senses using definition generation with open language models",
    author = "Fedorova, Mariia  and
      Kutuzov, Andrey  and
      Periti, Francesco  and
      Scherrer, Yves",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.findings-emnlp.1214/",
    pages = "22294--22302",
    ISBN = "979-8-89176-335-7",
    abstract = "We apply definition generators based on open-weights large language models to the task of creating explanations of novel senses, taking target word usages as an input. To this end, we employ the datasets from the AXOLOTL{'}24 shared task on explainable semantic change modeling, which features Finnish, Russian and German languages. We fine-tune and provide publicly the open-source models performing higher than the best submissions of the aforementioned shared task, which employed closed proprietary LLMs. In addition, we find that encoder-decoder definition generators perform on par with their decoder-only counterparts."
}

Framework versions

bert-score==0.3.13
peft==0.14.0
sentencepiece==0.2.0
tokenizers==0.20.1
torch==2.2.2
transformers==4.46.1
trl==0.15.2