finetuned-sileod-deberta-wic

DeBERTa-v3-large Fine-tuned on WiC

Cross-encoder model for the Word-in-Context (WiC) binary sense disambiguation task. Both sentences are fed together so the model can attend across them simultaneously.

Target Word Marking

The target word is wrapped with <tgt>word</tgt> in both sentences before encoding.

Performance

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Deehan1866/finetuned-sileod-deberta-wic")
model = AutoModelForSequenceClassification.from_pretrained("Deehan1866/finetuned-sileod-deberta-wic")

word = "bank"
s1 = f"The <tgt>{word}</tgt> raised its interest rates."
s2 = f"She visited her local <tgt>{word}</tgt> to deposit a cheque."

enc = tokenizer(s1, s2, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**enc).logits
pred = torch.argmax(logits).item()
print("Same sense" if pred == 1 else "Different sense")