persuasive_essays_distilbert_uncased

persuasive_essays_distilbert_uncased

Model description

This model is a fine-tuned version of distilbert-base-uncased on the emnlp2017-claim-identification/persuasive_essays dataset. It achieves the following results on the evaluation set:

• Loss: 0.4314
• Accuracy: 0.8001
• Macro F1: 0.7510
• Claim F1: 0.6403

Intended uses & limitations

Text classification for claims on full sentences. The model perfoms better at in-domain classification. Cross-domain classification is severely limited.

Training and evaluation data

Based on Stab and Gurevych (2017) persuasive essays corpus, preprocessed by [Daxenberger et al. (2017)]((https://github.com/UKPLab/emnlp2017-claim-identification).

Original dataset

• docs: 402
• tokens: 147,271
• total instances: 7,116 (65 duplicates)
  • #claims: 2,108 (29.62%)

Trimmed datast used for training

• total instances: 7051 (65 duplicates removed)
  • #claims: 2093 (29.68%)
• train/test split: 80/20, stratified

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0
• Datasets 2.17.0
• Tokenizers 0.15.2