Downloads · 30 days
28
19% of all-time downloads
pszemraj/xtremedistil-l12-h384-uncased-zeroshot-v1.1
xtremedistil-l12-h384-uncased-zeroshot-v1.1 is a zero-shot classification model from pszemraj. Use it when you need labels you did not train the model on. It is set up for transformers. The card lists the license as mit.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
28
19% of all-time downloads
All-time downloads
147
Public
Parameters
33.4M
133 MB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors133 MB · 99%
From the Hugging Face model README
A slightly larger sibling to https://hf.co/MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33
This model is a fine-tuned version of microsoft/xtremedistil-l12-h384-uncased on the None dataset. It achieves the following results on the evaluation set:
See https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/datasets_overview.csv
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Micro | Accuracy Balanced | Accuracy | Precision Macro | Recall Macro | Precision Micro | Recall Micro |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.2756 | 0.32 | 5000 | 0.4155 | 0.8146 | 0.8255 | 0.8215 | 0.8255 | 0.8101 | 0.8215 | 0.8255 | 0.8255 |
| 0.2395 | 0.65 | 10000 | 0.4166 | 0.8182 | 0.8303 | 0.8222 | 0.8303 | 0.8151 | 0.8222 | 0.8303 | 0.8303 |
| 0.2464 | 0.97 | 15000 | 0.4114 | 0.8204 | 0.8325 | 0.8239 | 0.8325 | 0.8175 | 0.8239 | 0.8325 | 0.8325 |
| 0.2105 | 1.3 | 20000 | 0.4051 | 0.8236 | 0.8363 | 0.8254 | 0.8363 | 0.8219 | 0.8254 | 0.8363 | 0.8363 |
| 0.2267 | 1.62 | 25000 | 0.4030 | 0.8244 | 0.8373 | 0.8257 | 0.8373 | 0.8231 | 0.8257 | 0.8373 | 0.8373 |
| 0.2312 | 1.95 | 30000 | 0.4088 | 0.8233 | 0.836 | 0.8250 | 0.836 | 0.8217 | 0.8250 | 0.836 | 0.836 |
| 0.2241 | 2.27 | 35000 | 0.4061 | 0.8257 | 0.8375 | 0.8291 | 0.8375 | 0.8229 | 0.8291 | 0.8375 | 0.8375 |
| 0.2183 | 2.6 | 40000 | 0.4043 | 0.8259 | 0.838 | 0.8285 | 0.838 | 0.8235 | 0.8285 | 0.838 | 0.838 |
| 0.2285 | 2.92 | 45000 | 0.4041 | 0.8241 | 0.8365 | 0.8263 | 0.8365 | 0.8220 | 0.8263 | 0.8365 | 0.8365 |