metalingo-indirect-metaphor

MetaLingo-Indirect-Metaphor

This model is a fine-tuned version of Microsoft DeBERTa-v3-large for binary indirect metaphor detection in English at token level, trained on the VU Amsterdam Metaphor Corpus (VUAMC). It is produced by a two-stage knowledge distillation pipeline: Stage 1 transfers knowledge from an auxiliary teacher model via soft-label distillation on an out-of-domain reference corpus (BE06, ~954K tokens), and Stage 2 fine-tunes on the VUAMC gold standard with hard labels. Annotation follows MIPVU (Steen et al. 2010); only indirect metaphors (metaphor_type == "met") are labeled positive.

Model description

• Base model: microsoft/deberta-v3-large
• Task: Token classification (binary metaphor detection)
• Training unit: Sentences, word-level labels
• Teacher model (Stage 1): An auxiliary DeBERTa-v3-large indirect-metaphor classifier used to produce soft labels for Stage 1 distillation; not released as a standalone model.

Two-stage distillation pipeline

Stage 1 — Distillation on BE06 (~954K tokens). The student is pre-trained with KL-divergence distillation (temperature = 2.0) against soft labels produced by an auxiliary DeBERTa-v3-large teacher specialized for indirect-metaphor detection. All Stage 1 training happens on BE06, a British-English reference corpus that is completely independent of VUAMC (different source corpus, different documents, zero overlap). This stage exposes the student to a much larger and more varied token distribution than VUAMC alone provides, giving it a strong initialization for metaphor detection before it ever sees a VUAMC sentence.

Stage 2 — Gold fine-tuning on VUAMC (90 train docs). Starting from the Stage 1 checkpoint, the model is fine-tuned with hard-label cross-entropy on the 90-document official VUAMC training split (10% randomly held out as dev, seed=42), then evaluated once on the 27-document official test split.

Training & evaluation data

• Dataset: VUAMC (VU Amsterdam Metaphor Corpus), covering four registers: News, Academic, Fiction, Conversation.
• Split: Official NAACL FLP partition (Leong et al. 2018): 90 train documents / 27 test documents. A random 10% of training sentences is held out as dev set (seed=42).
• Test set: 27 documents, 4,080 sentences, 57,811 tokens, 6,697 gold metaphor tokens.

Training hyperparameters

Results

Evaluated on the official 27-document test split (NAACL FLP 2018):

By genre

Genre is determined by BNC document ID per the NAACL FLP 2018 shared-task split (News: a*; Academic: b17, clw, cty, ecv; Fiction: bmw, ccw, faj; Conversation: k*).

By part of speech (Universal POS)

Tags with no gold metaphors in the test set (CCONJ, INTJ, NUM, SPACE, SYM, X) are omitted.

Label dictionary

{
  "0": "non_metaphor",
  "1": "metaphor"
}

Subwords are aligned to words via the tokenizer's word_ids; the first subword of each word is used for prediction.

Usage example

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_path = "tommyleo2077/metalingo-indirect-metaphor"  # or local path
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForTokenClassification.from_pretrained(model_path)
model.eval()

words = ["The", "conference", "threw", "a", "spanner", "in", "the", "works", "."]
inputs = tokenizer(
    words,
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=192,
)
word_ids = inputs.word_ids(batch_index=0)

with torch.no_grad():
    logits = model(**inputs).logits
preds = logits.argmax(dim=-1)[0].tolist()

word_preds = {}
for i, wid in enumerate(word_ids):
    if wid is not None and wid not in word_preds:
        word_preds[wid] = preds[i]

for i, w in enumerate(words):
    label = "metaphor" if word_preds.get(i, 0) == 1 else "non_metaphor"
    print(f"{w}\t{label}")

Citation

Model author: Tommy Leo — [email protected]

Dataset (MIPVU / VUAMC):

@book{steen2010method,
  title     = {A Method for Linguistic Metaphor Identification: From {MIP} to {MIPVU}},
  author    = {Steen, Gerard and Dorst, Aletta G. and Herrmann, J. Berenike and Kaal, Anna and Krennmayr, Tina and Pasma, Thea},
  year      = {2010},
  publisher = {John Benjamins}
}

Train/test split:

@inproceedings{leong2018vua,
  title     = {A Report on the 2018 {VUA} Metaphor Detection Shared Task},
  author    = {Leong, Chee Wee and Beigman Klebanov, Beata and Shutova, Ekaterina},
  booktitle = {Proceedings of the Workshop on Figurative Language Processing at NAACL-HLT 2018},
  year      = {2018}
}

Base model: Microsoft DeBERTa

This model:

@misc{leo2025metalingoindirectmetaphor,
  title        = {metalingo-indirect-metaphor: Two-Stage Knowledge Distillation for Metaphor Detection},
  author       = {Leo, Tommy},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/tommyleo2077/metalingo-indirect-metaphor}},
  note         = {Contact: [email protected]}
}

License

Apache License 2.0 — see LICENSE for details.