Downloads · 30 days
81
1% of all-time downloads
stacked-summaries/flan-t5-large-stacked-samsum-1024
flan-t5-large-stacked-samsum-1024 is a summarization model from stacked-summaries. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as apache-2.0.
<a href="https://colab.research.google.com/gist/pszemraj/a4bf61f593ebda9a8db6dc58839d9de4/brief-demo-flan-t5-stacked-samsum.ipynb" <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab…
Downloads · 30 days
81
1% of all-time downloads
All-time downloads
11.6K
Public
Parameters
783M
39.2 GB on disk
Likes
10
Public
Click a slice to open those files.
.bin3.1 GB · 33%
From the Hugging Face model README
This model is a fine-tuned version of google/flan-t5-large on the stacked-summaries/stacked-samsum-1024 dataset.
It achieves the following results on the evaluation set:
This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task-oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.
The token used to identify a new concept in the summary is [NEXT_CONCEPT]. You can split an output summary based on this token to see how it split the input text information: summary_text.split("[NEXT_CONCEPT]") etc.
samsum is not exactly the best dataset for general-purpose summarizationSee the dataset card linked on this page for info
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 0.1195 | 0.17 | 20 | 2.0635 | 57.8829 | 28.7887 | 44.4256 | 54.1299 | 121.8 |
| 0.1084 | 0.35 | 40 | 2.1178 | 58.0416 | 28.6487 | 44.3905 | 54.1557 | 122.893 |
| 0.1019 | 0.52 | 60 | 2.1576 | 57.816 | 28.7069 | 44.4242 | 53.9598 | 120.524 |
| 0.0975 | 0.7 | 80 | 2.1821 | 57.9597 | 28.8178 | 44.4854 | 54.068 | 121.793 |
| 0.0947 | 0.87 | 100 | 2.1846 | 57.9637 | 28.7446 | 44.3826 | 54.0399 | 122.77 |