Downloads · 30 days
216
0% of all-time downloads
MiniLMv2-toxic-jigsaw
minuva/MiniLMv2-toxic-jigsaw
MiniLMv2-toxic-jigsaw is a text classification model from minuva. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a fined-tuned version of MiniLMv2-L6-H384 on the on the Jigsaw 1st Kaggle competition dataset using unitary/toxic-bert as teacher model.
All-time downloads
79.9K
Public
Parameters
22.7M
364 MB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.pt182 MB · 50%
At a glance
- Task
- Text Classification
- Library
- transformers
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Dec 15, 2023
- Updated
- Feb 13, 2024
- SHA
- 00eacca7
Try a prompt
Model card
From the Hugging Face model README
Text Classification GoEmotions
This model is a fined-tuned version of MiniLMv2-L6-H384 on the on the Jigsaw 1st Kaggle competition dataset using unitary/toxic-bert as teacher model. The quantized version in ONNX format can be found here.
The model with two labels only (toxicity and severe toxicity) is here
Load the Model
from transformers import pipeline
pipe = pipeline(model='minuva/MiniLMv2-toxic-jigsaw', task='text-classification')
pipe("This is pure trash")
# [{'label': 'toxic', 'score': 0.9383478164672852}]
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 6e-05
- train_batch_size: 48
- eval_batch_size: 48
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- warmup_ratio: 0.1
Metrics (comparison with teacher model)
| Teacher (params) | Student (params) | Set (metric) | Score (teacher) | Score (student) |
|---|---|---|---|---|
| unitary/toxic-bert (110M) | MiniLMv2-toxic-jigsaw (23M) | Test (ROC_AUC) | 0.98636 | 0.98600 |
Deployment
Check out fast-nlp-text-toxicity repository for a FastAPI based server to deploy this model in CPU devices.
- Task
- Text Classification
- Library
- transformers
- Type
- bert
- License
- apache-2.0
- Languages
- en
- Created
- Dec 15, 2023
- Updated
- Feb 13, 2024