Downloads · 30 days
859
31% of all-time downloads
ModernBERT-Large-Instruct-WSD
sign/ModernBERT-Large-Instruct-WSD
ModernBERT-Large-Instruct-WSD is a machine learning model from sign. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Word sense disambiguation as multiple choice over WordNet (omw-en:1.4) definitions with a masked-LM answer slot. Used by https://github.com/sign/word-sense-disambiguation (the WSDModernBertForMaskedLM class there adds…
All-time downloads
2.8K
Public
Parameters
396M
4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors792 MB · 100%
At a glance
- License
- apache-2.0
- Model type
- modernbert
- Access
- Public
- Created
- Nov 23, 2025
- Updated
- Sep 4, 2026
- SHA
- e26867fb
Base models
Model card
From the Hugging Face model README
ModernBERT-Large-Instruct-WSD
Word sense disambiguation as multiple choice over WordNet (omw-en:1.4) definitions with a masked-LM answer
slot. Used by https://github.com/sign/word-sense-disambiguation (the WSDModernBertForMaskedLM class there
adds a 128-way answer-letter decoder; answer_letters.json lists the letters).
This revision (2026-09-04, "c3")
Fine-tuned from answerdotai/ModernBERT-Large-Instruct on: generated sentences for 97k synsets, WordNet's own
example sentences (minus a 5,000-example held-out slice), SemCor (222k instances, detokenized) and the
Princeton WordNet Gloss Corpus (manual tags). Cosine schedule, label smoothing 0.1, weight decay 0.01,
lr 2e-5, 2 epochs, batch 64, fp32 weights with bf16 autocast. Weights are stored in bf16.
Trained with the compact prompt template (the sentence with the target word marked, one definition per option
letter, a "none of the above" letter, then [unused0] [MASK]; 16% fewer tokens and ~20% faster than the earlier
wording). The repository code from PR #49 on builds exactly this prompt; earlier code/models are not compatible.
| Benchmark | previous revision (w4, earlier template) | this revision (c3) |
|---|---|---|
| WordNet held-out slice (5,000 examples, seed 42) | 78.6% | 78.3% |
| SemEval "ALL" (Senseval-2/3, SemEval-07/13/15; 7,247 instances, any gold key) | 80.7% | 80.6% |
| Throughput, one H100, same pipeline | 3,614 prompts/s | 4,326 prompts/s |
The previous revision over-predicted "none of the above" on long natural sentences; this one almost never does.
Full recipe and sweep configs: training/ in the repository.
- Type
- modernbert
- License
- apache-2.0
- Languages
- en
- Created
- Nov 23, 2025
- Updated
- Sep 4, 2026