finetuned-distilbert-gibberish-legit-classifier

DistilBERT Gibberish vs Legit Classifier

Task

This model classifies short English text into two classes:

• 0 = gibberish
• 1 = legit

The goal is to predict whether the text is readable or gibberish.

Dataset

Dataset: pakiino/2026-24679-gibberish-legit-text-hw1

The original train, validation, and test splits were used.

Only text was used as the input feature. label was used as the target.

Other columns such as source ID and augmentation information were not used as model inputs.

Preprocessing

The text was processed automatically by AutoGluon MultiModal.

The maximum text length was 128 tokens.

Model

Base model: distilbert-base-uncased

The model was fine-tuned with AutoGluon MultiModal.

Fine-tuning mode: norm_fit

Training

• Epochs: 5
• Learning rate: 2e-5
• Batch size: 8
• Random seed: 24679
• Evaluation metric: accuracy

The validation set was used to select the best model.

Results

Test Accuracy: 1.0000

Weighted F1: 1.0000

A confusion matrix was also used to evaluate the model.

Error Analysis

There were no misclassified examples in the test set.

The model correctly classified all test examples, so there were no specific errors to analyze.

However, the test set is small, so the result may not represent performance on other types of text.

Limitations

The dataset is small and mainly contains engineering workshop text.

The model may not perform as well on text from other topics.

Gibberish examples use similar vocabulary to the legit examples, so the model may rely on grammar and word order.

Ethics

It should not be used to make important decisions about people.

Hardware / Compute

The model was trained in Google Colab using GPU acceleration.

License

The base DistilBERT model uses the Apache 2.0 license.

Please also refer to the original dataset page for its license information.

AI Usage

Generative AI was used as a support tool to help explain concepts, troubleshoot code, and improve documentation.

I selected the dataset, designed the code framenwork, reviewed the results, and created the final model.