Downloads · 30 days
19
4% of all-time downloads
andreiujica/led-base-big-patent
led-base-big-patent is a summarization model from andreiujica. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
19
4% of all-time downloads
All-time downloads
502
Public
Parameters
162M
38.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors648 MB · 99%
From the Hugging Face model README
This model is a fine-tuned version of allenai/led-base-16384 on the big_patent dataset. It achieves the following results on the evaluation set:
This model is a long-form text generation model based on the Longformer Encoder-Decoder (LED) architecture. The LED architecture extends the Transformer model to handle long documents by incorporating sparse attention mechanisms. This makes it suitable for tasks such as summarization of lengthy patent documents, where traditional models might struggle with context length limitations. The model has been fine-tuned on the BigPatent dataset, a large collection of patent documents, to enhance its performance in generating concise and informative summaries.
Intended uses
Limitations
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 0.2195 | 1.0 | 9595 | 0.2350 | 0.2858 | 0.1157 | 0.2372 | 0.237 | 50.5544 |
| 0.1754 | 2.0 | 19191 | 0.2363 | 0.2895 | 0.1172 | 0.2392 | 0.2389 | 49.5847 |
| 0.1428 | 3.0 | 28785 | 0.2412 | 0.2972 | 0.1193 | 0.2431 | 0.243 | 46.326 |