mnli_model_deberta_3_labels

Based on svenbl80/deberta-v3-Base-finetuned-mnli finetuned on a synthetic dataset (labels)

Performance on test dataset: precision recall f1-score support

       0       0.99      1.00      0.99        94
       1       1.00      1.00      1.00        28
       2       1.00      0.98      0.99        66

accuracy                           0.99       188
macro avg       1.00      0.99      1.00       188
weighted avg       0.99      0.99      0.99       188

Performance on real estate benchmark: precision recall f1-score support

       0       0.30      0.45      0.36       100
       1       0.21      0.15      0.18       100
       2       0.35      0.27      0.31       100

accuracy                           0.29       300
macro avg       0.29      0.29      0.28       300
weighted avg       0.29      0.29      0.28       300

Baseline (svenbl80/deberta-v3-Base-finetuned-mnli) for real estate benchmark:

       0       0.89      0.68      0.77       100
       1       0.63      0.92      0.75       100
       2       0.88      0.69      0.78       100

accuracy                           0.76       300
macro avg       0.80      0.76      0.77       300
weighted avg       0.80      0.76      0.77       300