Downloads · 30 days
10
32% of all-time downloads
longformer-bert-base-4096
Deok-gi/longformer-bert-base-4096
longformer-bert-base-4096 is a machine learning model from Deok-gi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
longformer-bert-base-4096 is a model that started from the BERT checkpoint and was pretrained for MLM on long documents. It supports sequences of length up to 4,096.
All-time downloads
31
Public
Repo size
1.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin808 MB · 100%
At a glance
- License
- apache-2.0
- Model type
- bert
- Access
- Public
- Created
- Nov 15, 2025
- Updated
- Nov 15, 2025
- SHA
- 2b062af2
Model card
From the Hugging Face model README
longformer-bert-base-4096
longformer-bert-base-4096 is a model that started from the BERT checkpoint and was pretrained for MLM on long documents. It supports sequences of length up to 4,096.
Citing
If you use Longformer in your research, please cite Longformer: The Long-Document Transformer.
@article{Beltagy2020Longformer,
title={Longformer: The Long-Document Transformer},
author={Iz Beltagy and Matthew E. Peters and Arman Cohan},
journal={arXiv:2004.05150},
year={2020},
}
Longformer is an open-source project developed by the Allen Institute for Artificial Intelligence (AI2).
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
- Type
- bert
- License
- apache-2.0
- Created
- Nov 15, 2025
- Updated
- Nov 15, 2025