Downloads · 30 days
10
1% of all-time downloads
distilbert-base-uncased-finetuned-ag-news-v5
miggwp/distilbert-base-uncased-finetuned-ag-news-v5
distilbert-base-uncased-finetuned-ag-news-v5 is a fill-mask model from miggwp. Use it when you need the model to fill a missing word. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
766
Public
Parameters
67M
41 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors268 MB · 100%
At a glance
- Task
- Fill-Mask
- Library
- transformers
- License
- apache-2.0
- Model type
- distilbert
- Access
- Public
- Created
- Aug 4, 2024
- Updated
- Aug 25, 2024
- SHA
- 6d45a752
Try a prompt
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
distilbert-base-uncased-finetuned-ag-news-v5
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.6547
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 55
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.6805 | 1.0 | 1384 | 2.3077 |
| 2.322 | 2.0 | 2768 | 2.1636 |
| 2.2131 | 3.0 | 4152 | 2.0745 |
| 2.1394 | 4.0 | 5536 | 2.0587 |
| 2.0808 | 5.0 | 6920 | 1.9837 |
| 2.0311 | 6.0 | 8304 | 1.9698 |
| 1.9923 | 7.0 | 9688 | 1.9465 |
| 1.9625 | 8.0 | 11072 | 1.9263 |
| 1.9321 | 9.0 | 12456 | 1.9020 |
| 1.9055 | 10.0 | 13840 | 1.8654 |
| 1.8754 | 11.0 | 15224 | 1.8343 |
| 1.8593 | 12.0 | 16608 | 1.8507 |
| 1.838 | 13.0 | 17992 | 1.8369 |
| 1.8132 | 14.0 | 19376 | 1.8325 |
| 1.8026 | 15.0 | 20760 | 1.8055 |
| 1.785 | 16.0 | 22144 | 1.8030 |
| 1.7709 | 17.0 | 23528 | 1.7813 |
| 1.7559 | 18.0 | 24912 | 1.7845 |
| 1.7378 | 19.0 | 26296 | 1.7500 |
| 1.7289 | 20.0 | 27680 | 1.7456 |
| 1.7171 | 21.0 | 29064 | 1.7617 |
| 1.7061 | 22.0 | 30448 | 1.7580 |
| 1.6923 | 23.0 | 31832 | 1.7395 |
| 1.6889 | 24.0 | 33216 | 1.7297 |
| 1.6685 | 25.0 | 34600 | 1.7360 |
| 1.6649 | 26.0 | 35984 | 1.7113 |
| 1.6538 | 27.0 | 37368 | 1.7019 |
| 1.6511 | 28.0 | 38752 | 1.6995 |
| 1.6358 | 29.0 | 40136 | 1.7171 |
| 1.6328 | 30.0 | 41520 | 1.7231 |
| 1.6223 | 31.0 | 42904 | 1.6945 |
| 1.6157 | 32.0 | 44288 | 1.7166 |
| 1.6129 | 33.0 | 45672 | 1.6882 |
| 1.6062 | 34.0 | 47056 | 1.7005 |
| 1.5989 | 35.0 | 48440 | 1.7029 |
| 1.5969 | 36.0 | 49824 | 1.7047 |
| 1.5909 | 37.0 | 51208 | 1.6756 |
| 1.5766 | 38.0 | 52592 | 1.6903 |
| 1.5764 | 39.0 | 53976 | 1.6645 |
| 1.5757 | 40.0 | 55360 | 1.6601 |
| 1.5662 | 41.0 | 56744 | 1.6500 |
| 1.5637 | 42.0 | 58128 | 1.6824 |
| 1.554 | 43.0 | 59512 | 1.6766 |
| 1.5599 | 44.0 | 60896 | 1.6728 |
| 1.5529 | 45.0 | 62280 | 1.6640 |
| 1.5503 | 46.0 | 63664 | 1.6706 |
| 1.5459 | 47.0 | 65048 | 1.6596 |
| 1.5388 | 48.0 | 66432 | 1.6676 |
| 1.5425 | 49.0 | 67816 | 1.6559 |
| 1.5378 | 50.0 | 69200 | 1.6551 |
| 1.5365 | 51.0 | 70584 | 1.6647 |
| 1.5347 | 52.0 | 71968 | 1.6293 |
| 1.5356 | 53.0 | 73352 | 1.6551 |
| 1.527 | 54.0 | 74736 | 1.6563 |
| 1.5326 | 55.0 | 76120 | 1.6439 |
Framework versions
- Transformers 4.42.4
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
- Task
- Fill-Mask
- Library
- transformers
- Type
- distilbert
- License
- apache-2.0
- Created
- Aug 4, 2024
- Updated
- Aug 25, 2024