news_classification_model

news_classification_model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0865
• Train Accuracy: 0.9712
• Validation Loss: 1.5070
• Validation Accuracy: 0.7252
• Epoch: 1

Model description

Output Labels of this Model are:

• 'HEALTH & MEDICINE': 0,
• 'ENTERTAINMENT': 1,
• 'FINANCE': 2,
• 'SCIENCE & TECHNOLOGY': 3,
• 'WORLD NEWS': 4,
• 'EDUCATION': 5,
• 'SOCIETY': 6,
• 'POLITICS': 7,
• 'ENVIRONMENT': 8,
• 'BUSINESS': 9,
• 'MEDIA': 10,
• 'FOOD': 11,
• 'CRIME': 12,
• 'LIFESTYLE': 13,
• 'SPORTS': 14

Intended uses & limitations

This models classifies News headlines into different categories mentioned above.

Training and evaluation data

Model was trained and evaluated using the dataset available here: https://www.kaggle.com/datasets/rmisra/news-category-dataset

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam',

• 'weight_decay': None,

• 'clipnorm': None,

• 'global_clipnorm': None,

• 'clipvalue': None,

• 'use_ema': False,

• 'ema_momentum': 0.99,

• 'ema_overwrite_frequency': None,

• 'jit_compile': True,

• 'is_legacy_optimizer': False,

• 'learning_rate': 5e-05,

• 'beta_1': 0.9,

• 'beta_2': 0.999,

• 'epsilon': 1e-07,

• 'amsgrad': False}

• training_precision: float32

Training results

Upon the second round of trainning, the results were as follows:

Framework versions

• Transformers 4.33.1
• TensorFlow 2.13.0
• Tokenizers 0.13.3