bot-detection-indobert

IndoBERT for Bot Detection on Platform X

This model uses IndoBERT to classify Twitter/X accounts as bots or humans based on their textual content and numeric features.

Model Architecture

• Base Model: indobenchmark/indobert-base-p1

Dataset

The dataset includes Indonesian tweets with labels (0 for human, 1 for bot), along with additional numeric features such as favorite_count, retweet_count, reply_count, and quote_count.

Training Details

• Loss Function: Binary Cross Entropy (BCELoss)

• Optimizer: AdamW (lr = 2e-5)

• Epochs: 11

• Max Token Length: 128

• Batch Size: 16

• Epochs: 11 | Epoch | Train Loss | Val Accuracy | Precision | Recall | F1-score | |-------|------------|--------------|-----------|--------|----------| | 1 | 0.3683 | 0.8311 | 0.8511 | 0.8046 | 0.8272 | | 2 | 0.2532 | 0.8940 | 0.9138 | 0.8712 | 0.8920 | | 3 | 0.1469 | 0.9431 | 0.9513 | 0.9347 | 0.9429 | | 4 | 0.0816 | 0.9687 | 0.9709 | 0.9667 | 0.9688 | | 5 | 0.0534 | 0.9800 | 0.9821 | 0.9780 | 0.9801 | | 6 | 0.0462 | 0.9830 | 0.9846 | 0.9815 | 0.9831 | | 7 | 0.0345 | 0.9868 | 0.9869 | 0.9868 | 0.9869 | | 8 | 0.0357 | 0.9870 | 0.9878 | 0.9863 | 0.9871 | | 9 | 0.0303 | 0.9891 | 0.9902 | 0.9881 | 0.9891 | | 10 | 0.0245 | 0.9911 | 0.9915 | 0.9908 | 0.9912 | | 11 | 0.0245 | 0.9911 | 0.9915 | 0.9908 | 0.9912 |

Evaluation Results

Final Validation Accuracy: 0.8600

Final Precision: 0.8459 Final Recall: 0.8790 Final F1-score: 0.8621

Confusion Matrix

Training Phase

Classification Report

              precision    recall  f1-score   support

         0.0       0.88      0.84      0.86      2008
         1.0       0.85      0.88      0.86      1992

    accuracy                           0.86      4000
   macro avg       0.86      0.86      0.86      4000
weighted avg       0.86      0.86      0.86      4000