Downloads · 30 days
16
31% of all-time downloads
bart_bertsum_1024_375_1000
jordanfan/bart_bertsum_1024_375_1000
bart_bertsum_1024_375_1000 is a machine learning model from jordanfan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
51
Public
Parameters
406M
4.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 GB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Model type
- bart
- Access
- Public
- Created
- Mar 20, 2024
- Updated
- Mar 20, 2024
- SHA
- 48dfe869
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
bart_bertsum_1024_375_1000
This model is a fine-tuned version of facebook/bart-large on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.0535
- Rouge1: 0.6801
- Rouge2: 0.4119
- Rougel: 0.6159
- Rougelsum: 0.616
- Wer: 0.4729
- Bleurt: -0.3664
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 6
- eval_batch_size: 6
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Wer | Bleurt |
|---|---|---|---|---|---|---|---|---|---|
| No log | 0.13 | 250 | 1.2919 | 0.636 | 0.3519 | 0.567 | 0.567 | 0.5296 | -0.0182 |
| 2.2326 | 0.27 | 500 | 1.2002 | 0.6503 | 0.3707 | 0.5816 | 0.5817 | 0.5113 | -0.7073 |
| 2.2326 | 0.4 | 750 | 1.1735 | 0.6564 | 0.3791 | 0.5898 | 0.5898 | 0.5048 | -0.3421 |
| 1.2886 | 0.53 | 1000 | 1.1476 | 0.661 | 0.3843 | 0.594 | 0.5939 | 0.4994 | 0.0835 |
| 1.2886 | 0.66 | 1250 | 1.1289 | 0.6615 | 0.3863 | 0.5938 | 0.5938 | 0.4945 | -0.5247 |
| 1.2306 | 0.8 | 1500 | 1.1197 | 0.67 | 0.3952 | 0.6046 | 0.6045 | 0.4909 | -0.192 |
| 1.2306 | 0.93 | 1750 | 1.1077 | 0.6734 | 0.3989 | 0.6068 | 0.6067 | 0.4876 | -0.3867 |
| 1.1852 | 1.06 | 2000 | 1.0917 | 0.6731 | 0.4027 | 0.609 | 0.609 | 0.4833 | -0.6453 |
| 1.1852 | 1.2 | 2250 | 1.0852 | 0.6707 | 0.4013 | 0.6054 | 0.6054 | 0.4824 | -0.5589 |
| 1.0875 | 1.33 | 2500 | 1.0785 | 0.6738 | 0.4049 | 0.6096 | 0.6096 | 0.4794 | -0.5107 |
| 1.0875 | 1.46 | 2750 | 1.0709 | 0.6743 | 0.4046 | 0.6096 | 0.6095 | 0.478 | -0.3387 |
| 1.0857 | 1.6 | 3000 | 1.0627 | 0.6778 | 0.41 | 0.6137 | 0.6137 | 0.4757 | -0.4275 |
| 1.0857 | 1.73 | 3250 | 1.0636 | 0.675 | 0.4088 | 0.6121 | 0.612 | 0.4745 | -0.3664 |
| 1.0634 | 1.86 | 3500 | 1.0552 | 0.6775 | 0.4103 | 0.6136 | 0.6136 | 0.4729 | -0.3664 |
| 1.0634 | 1.99 | 3750 | 1.0535 | 0.6801 | 0.4119 | 0.6159 | 0.616 | 0.4729 | -0.3664 |
Framework versions
- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
- Library
- transformers
- Type
- bart
- License
- apache-2.0
- Created
- Mar 20, 2024
- Updated
- Mar 20, 2024