Downloads · 30 days
13
3% of all-time downloads
led-bartpho-word-base-16384
bluenguyen/led-bartpho-word-base-16384
led-bartpho-word-base-16384 is a machine learning model from bluenguyen. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
This model was initialized from vinai/bartpho-word-base and converted to Allenai's Longformer Encoder-Decoder (LED) based on Longformer: The Long-Document Transformer.
All-time downloads
457
Public
Repo size
1.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin690 MB · 99%
At a glance
- Library
- transformers
- Model type
- led
- Access
- Public
- Created
- Dec 7, 2022
- Updated
- Jan 11, 2023
- SHA
- 848d91ec
Model card
From the Hugging Face model README
Introduction
This model was initialized from vinai/bartpho-word-base and converted to Allenai's Longformer Encoder-Decoder (LED) based on Longformer: The Long-Document Transformer.
To be able to process 16K tokens, bartpho-word-base's position embedding matrix was simply copied 16 times.
This model is especially interesting for long-range summarization and question answering.
Fine-tuning for down-stream task
This notebook shows how led model can effectively be fine-tuned on a downstream task.
- Library
- transformers
- Type
- led
- Languages
- vi
- Created
- Dec 7, 2022
- Updated
- Jan 11, 2023