modernbert-bug-severity

ModernBERT Bug Severity Classifier

This model is a fully fine-tuned version of answerdotai/ModernBERT-base for classifying short bug descriptions into six severity levels:

• blocker
• critical
• major
• normal
• minor
• trivial

Intended use

This model is an educational demonstration of automated bug-severity classification.

It should not be used as the sole authority for production severity decisions. High-impact predictions should be reviewed by a human.

Training dataset

The model was trained using:

AliArshad/Bugzilla_Eclipse_Bug_Reports_Dataset

Only the Short Description field was used as the model input. The Severity Label field was used as the target label.

Base model

answerdotai/ModernBERT-base

Training approach

This model was trained using full fine-tuning. It is not a LoRA or adapter-only model.

Training included:

• Removing missing and empty descriptions
• Converting severity names into numeric labels
• Stratified training, validation, and test splits
• Batched tokenization
• Dynamic padding and attention masks
• Hugging Face Trainer
• Macro F1 checkpoint selection

Evaluation

• Test accuracy: 0.871
• Test macro F1: 0.32951261884727445

Performance should also be examined separately for each severity using the classification report and confusion matrix.

Limitations

• Severity cannot always be determined from a short description alone.
• The training dataset may contain noisy or inconsistent labels.
• The severity classes are imbalanced.
• Historical bug reports may not represent current software practices.
• Softmax confidence is not guaranteed to be a calibrated probability.
• Human review is recommended for blocker and critical predictions.

License status

The ModernBERT base model uses the Apache 2.0 license. The training dataset's Hugging Face page does not currently declare a dataset license. Confirm the applicable dataset and source-data terms before making this fine-tuned model public.

Example usage

Install the required library:

pip install transformers torch

Run inference:

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="abhishes/modernbert-bug-severity",
    revision="v1.0.0"
)

bug_reports = [
    "Application crashes immediately and all data is lost.",
    "There is a spelling mistake in the documentation."
]

results = classifier(bug_reports)

print(results)