Downloads · 30 days
0
Mortezaem/Sentiment_Analysis_PEFT_on_DistilBert
Sentiment_Analysis_PEFT_on_DistilBert is a machine learning model from Mortezaem. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
0
Access
Public
Updated Apr 18, 2024
Parameters
67.6M
270 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors270 MB · 100%
From the Hugging Face model README
This model is a fine-tuned version of distilbert-base-uncased on the sentiment140 (https://huggingface.co/datasets/sentiment140) dataset. Due to time and computational limits, I <b>used only 10% of the dataset</b> with a data augmentation. Despite minimal training, model's accuracy <b>improved by 59.5%</b> in comparison with non-fine tuned DistilBert LLM!<br>
It achieves the following results on the evaluation set:
<b>Classification Scores:</b><br>
Fine Tuned DistilBert LLM with LoRa: Accuracy = 0.78, Precision = 0.77, Recall = 0.78, F1 Score = 0.78
Non-Fine Tuned DistilBert LLM: Accuracy = 0.49, Precision = 0.45, Recall = 0.14, F1 Score = 0.22
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.5092 | 1.0 | 5688 | 0.4979 | 0.7591 |
| 0.4928 | 2.0 | 11376 | 0.4876 | 0.7662 |
| 0.4915 | 3.0 | 17064 | 0.4821 | 0.7711 |
| 0.4787 | 4.0 | 22752 | 0.4779 | 0.7731 |
| 0.4757 | 5.0 | 28440 | 0.4767 | 0.7746 |
| 0.473 | 6.0 | 34128 | 0.4743 | 0.775 |
| 0.4649 | 7.0 | 39816 | 0.4741 | 0.7751 |
| 0.4709 | 8.0 | 45504 | 0.4732 | 0.7753 |