Downloads · 30 days
10
2% of all-time downloads
mariav/distilbert-base-german-cased-finetuned-amazon-reviews
distilbert-base-german-cased-finetuned-amazon-reviews is a fill-mask model from mariav. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
10
2% of all-time downloads
All-time downloads
647
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.bin270 MB · 100%
From the Hugging Face model README
This model is a fine-tuned version of distilbert-base-german-cased on the amazon_reviews_multi dataset. It achieves the following results on the evaluation set:
This model is a fine-tuned version of distilbert-base-german-cased using the dataset from amazon_reviews_multi (available in Huggin Face). The purpose is to extend the model's domain, which, once fine-tuned, will be modified for the fill-in-the-gaps task. It's related to my other model (fine-tuned-spanish-bert) as a comparison of both performances.
The use is limited to school use and the limitations have to do with the size of the dataset, since it does not allow for a large contribution, a larger dataset would have to be used to get a larger contribution.
I did a training that gives the training and validation set loss. (It takes a lot of time. If you're using colab, I recommend to use less Epochs because the result does not change too much, and even though the loss is quite high, the performance of the model based on the perplexity is not that bad) Also, I checked the perplexity, which is one good measure for Languages Models. The value of the perplexity is considerabily good: 48'78.
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.625 | 1.0 | 141 | 4.2747 |
| 4.3013 | 2.0 | 282 | 4.1549 |
| 4.1841 | 3.0 | 423 | 4.0902 |
| 4.1208 | 4.0 | 564 | 3.9958 |
| 4.0475 | 5.0 | 705 | 3.9710 |
| 4.0116 | 6.0 | 846 | 3.9100 |
| 3.9988 | 7.0 | 987 | 3.9194 |
| 3.9641 | 8.0 | 1128 | 3.9381 |
| 3.9661 | 9.0 | 1269 | 3.8631 |
| 3.944 | 10.0 | 1410 | 3.8668 |