deberta-v3-large-metaphor

DeBERTa-v3-large-metaphor

This model is a fine-tuned version of Microsoft DeBERTa-v3-large for binary indirect metaphor detection in English at token level, trained on the VU Amsterdam Metaphor Corpus (VUAMC). Training uses sentence-level segmentation and a two-class scheme (non-metaphor / metaphor), following the MIPVU annotation guidelines (Steen et al. 2010). Only indirect metaphors (metaphor_type == "met") are labeled positive.

Model description

• Base model: microsoft/deberta-v3-large
• Task: Token classification (binary metaphor detection)
• Training unit: Full sentences, word-level labels
• Label scheme: binary — 0 = non_metaphor, 1 = metaphor

Training & evaluation data

• Dataset: VUAMC (VU Amsterdam Metaphor Corpus; Steen et al. 2010), covering four registers: News, Academic, Fiction, Conversation.
• Split: Official NAACL FLP partition (Leong et al. 2018): 90 train documents / 27 test documents. A random 10% of training sentences is held out as a development set (seed=42).
• Final splits: Train = 10,893 sentences · Dev = 1,210 sentences · Test = 4,080 sentences.

Test set genre distribution

Genre is determined by BNC-Baby document ID prefix, following the NAACL FLP 2018 shared-task mapping:

Contamination verification

Test document filenames are explicitly excluded from the training file list at load time. No test document appears in the training or development split.

Training hyperparameters

Results

Evaluated on the 27-document NAACL FLP test split (4,080 sentences, 57,811 tokens, 6,697 gold metaphor tokens):

Results by genre

Results by part of speech (Universal POS)

POS tags with zero gold metaphors in the test set are omitted.

Label dictionary

{
  "0": "non_metaphor",
  "1": "metaphor"
}

Subwords are aligned to words via the tokenizer's word_ids; the first subword of each word is used for prediction. Special tokens ([CLS], [SEP], [PAD]) and continuation subwords receive IGNORE_INDEX = -100 and are excluded from all metric calculations.

Usage example

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_path = "tommyleo2077/deberta-v3-large-metaphor"  # or local path
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForTokenClassification.from_pretrained(model_path)
model.eval()

words = ["The", "conference", "threw", "a", "spanner", "in", "the", "works", "."]
inputs = tokenizer(
    words,
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=192,
)
word_ids = inputs.word_ids(batch_index=0)

with torch.no_grad():
    logits = model(**inputs).logits
preds = logits.argmax(dim=-1)[0].tolist()

word_preds = {}
for i, wid in enumerate(word_ids):
    if wid is not None and wid not in word_preds:
        word_preds[wid] = preds[i]

for i, w in enumerate(words):
    label = "metaphor" if word_preds.get(i, 0) == 1 else "non_metaphor"
    print(f"{w}\t{label}")

Citation

Model author: Tommy Leo — [email protected]

Dataset (MIPVU / VUAMC):

@book{steen2010method,
  title     = {A Method for Linguistic Metaphor Identification: From {MIP} to {MIPVU}},
  author    = {Steen, Gerard and Dorst, Aletta G. and Herrmann, J. Berenike and Kaal, Anna and Krennmayr, Tina and Pasma, Thea},
  year      = {2010},
  publisher = {John Benjamins}
}

Train/test split:

@inproceedings{leong2018vua,
  title     = {A Report on the 2018 {VUA} Metaphor Detection Shared Task},
  author    = {Leong, Chee Wee and Beigman Klebanov, Beata and Shutova, Ekaterina},
  booktitle = {Proceedings of the Workshop on Figurative Language Processing at NAACL-HLT 2018},
  year      = {2018}
}

Base model: Microsoft DeBERTa

This model:

@misc{leo2025debertav3largemetaphor,
  title        = {deberta-v3-large-metaphor: Sentence-Level Fine-Tuning for Metaphor Detection},
  author       = {Leo, Tommy},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/tommyleo2077/deberta-v3-large-metaphor}},
  note         = {Contact: [email protected]}
}