bert-base-uncased-finetuned-glue-sst2

bert-base-uncased-finetuned-glue-sst2

Use for sentiment analysis.

This model is a fine-tuned version of bert-base-uncased on the glue sst2 dataset. The model achieves 91.39% accuracy on the validation dataset.

Model description

bert-base-uncased is a pretrained English language model. bert-base-uncased-finetuned-glue-sst2 adds a 2-class classification head for predicting positive and negative sentiment.

Training and evaluation data

The model has been trained on 10K training samples, even though the glue sst2 dataset contains 67.3K samples. This was done to decrease training time.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 3750, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

• Accuracy (training): 94.33%
• Accuracy (validation): 91.39%

Framework versions

• Transformers 4.35.2
• TensorFlow 2.15.0
• Datasets 2.16.1
• Tokenizers 0.15.0