cefr-level-deberta-v3-base

DeBERTa-v3 CEFR Vocabulary Classifier

A fine-tuned DeBERTa-v3-base model for predicting the CEFR proficiency level of English vocabulary items.

The model classifies words into six Common European Framework of Reference (CEFR) levels:

• A1
• A2
• B1
• B2
• C1
• C2

This model is intended for:

• Vocabulary difficulty estimation
• Language learning applications
• CEFR-aware educational tools
• Vocabulary profiling
• Adaptive learning systems
• Linguistic research

---

Model Details

---

Dataset

This model was trained using CEFR annotations derived from:

Dataset: star092304/CEFR-Annotated-WordNet

Dataset card:

https://huggingface.co/datasets/star092304/CEFR-Annotated-WordNet

The dataset provides CEFR proficiency annotations for WordNet lexical entries and was created based on the following work:

Reference Paper

CEFR-Annotated WordNet: LLM-Based Proficiency-Guided Semantic Database for Language Learning

Authors:

• Masato Kikuchi
• Masatsugu Ono
• Toshioki Soga
• Tetsu Tanabe
• Tadachika Ozono

Paper:

https://arxiv.org/html/2510.18466v2

Citation

@article{kikuchi2025cefrannotatedwordnet,
  title={CEFR-Annotated WordNet: LLM-Based Proficiency-Guided Semantic Database for Language Learning},
  author={Kikuchi, Masato and Ono, Masatsugu and Soga, Toshioki and Tanabe, Tetsu and Ozono, Tadachika},
  year={2025}
}

---

Training Data Construction

Training examples were generated by aligning:

• SemCor sense annotations
• WordNet lexical entries
• CEFR labels from CEFR-Annotated WordNet

Additional preprocessing included:

• Lemmatization
• Sense matching
• Multi-word expression removal
• Proper noun filtering

Only single-word lexical items were retained for training.

---

Training Performance

Training Curves

---

Test Confusion Matrix

---

Evaluation Preview

---

Usage

from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
import torch

model_name = "star092304/cefr-level-deberta-v3-base"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

label_names = ["A1", "A2", "B1", "B2", "C1", "C2"]

word = "investigation"

inputs = tokenizer(
    word,
    return_tensors="pt",
    truncation=True
)

with torch.no_grad():
    outputs = model(**inputs)

pred_id = outputs.logits.argmax(dim=-1).item()

print(label_names[pred_id])

---

Example Predictions

---

Limitations

• The model predicts vocabulary difficulty at the word level.
• CEFR levels can vary depending on context and meaning.
• Polysemous words may belong to multiple CEFR levels depending on usage.
• Predictions should be interpreted as estimated proficiency levels rather than absolute ground truth.

---

Intended Use

This model is intended for:

• Educational research
• Language learning systems
• Vocabulary recommendation engines
• CEFR-aware NLP pipelines

The model is not intended for high-stakes educational assessment or certification decisions.

---

Acknowledgements

This work builds upon:

• WordNet
• SemCor
• CEFR-Annotated WordNet
• Hugging Face Transformers
• Microsoft DeBERTa-v3

Special thanks to the authors of the CEFR-Annotated WordNet dataset and paper.