Downloads · 30 days
24
0% of all-time downloads
pszemraj/long-t5-tglobal-base-16384-booksci-summary-v1
long-t5-tglobal-base-16384-booksci-summary-v1 is a summarization model from pszemraj. Use it when you need a shorter version of a longer text. It is set up for transformers. The card lists the license as bsd-3-clause.
<a href="https://colab.research.google.com/gist/pszemraj/ee06b2b3bfcd7d29e588697a90f7a776/long-t5-tglobal-base-16384-booksci-summary-v1-example-with-textsum.ipynb" <img src="https://colab.research.google.com/assets/co…
Downloads · 30 days
24
0% of all-time downloads
All-time downloads
6.2K
Public
Parameters
248M
3 GB on disk
Likes
2
Public
Click a slice to open those files.
.onnx993 MB · 33%
From the Hugging Face model README
An experiment investigating transfer learning capabilities by fine-tuning models on different datasets starting from the booksum checkpoint.
This model is a fine-tuned version of pszemraj/long-t5-tglobal-base-16384-book-summary on the pszemraj/scientific_lay_summarisation-elife-norm dataset for two epochs.
It's recommended to use this model with beam search decoding. If interested, you can also use the textsum util repo to have most of this abstracted out for you:
pip install -U textsum
from textsum.summarize import Summarizer
model_name = "pszemraj/long-t5-tglobal-base-16384-booksci-summary-v1"
summarizer = Summarizer(model_name) # GPU auto-detected
text = "put the text you don't want to read here"
summary = summarizer.summarize_string(text)
print(summary)
Note: this model was trained at a lower LR & not till "absolute convergence" with the intention of retaining some of the properties learned from the initial fine-tuning on
booksum
It achieves the following results on the evaluation set:
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 2.7492 | 0.99 | 67 | 2.4272 | 34.6436 | 4.4536 | 12.4985 | 30.916 | 300.7635 |
| 2.6689 | 1.97 | 134 | 2.3994 | 34.2428 | 4.3644 | 12.5332 | 30.6965 | 294.0249 |