Downloads · 30 days
68
6% of all-time downloads
AlgorithmicResearchGroup/led_base_16384_billsum_summarization
led_base_16384_billsum_summarization is a summarization model from AlgorithmicResearchGroup. Use it when you need a shorter version of a longer text. It is set up for transformers.
This model is a fine-tuned version of led-base-16384 on the billsum dataset.
Downloads · 30 days
68
6% of all-time downloads
All-time downloads
1.1K
Public
Parameters
460M
7.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.pt3.7 GB · 50%
From the Hugging Face model README
This model is a fine-tuned version of led-base-16384 on the billsum dataset.
As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("Artifact-AI/led_base_16384_billsum_summarization")
model = AutoModelForSeq2SeqLM.from_pretrained("Artifact-AI/led_base_16384_billsum_summarization")
| Model | Rouge-1 | Rouge-2 | Rouge-L | Rouge-Lsum |
|---|---|---|---|---|
| LED Large | 47.843 | 26.342 | 34.230 | 41.689 |
| LED Base | 47.672 | 26.737 | 34.568 | 41.529 |
The model is trained on the BillSum summarization dataset found here
Please find a notebook to test the model below:
@misc{led_base_16384_billsum_summarization,
title={led_base_16384_billsum_summarization},
author={Matthew Kenney},
year={2023}
}