Downloads · 30 days
20
5% of all-time downloads
mamba-130m-pg19
assafbk/mamba-130m-pg19
mamba-130m-pg19 is a text generation model from assafbk. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
The official checkpoint of Mamba-130m, finetuned for Language Modeling over the PG-19 dataset as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
All-time downloads
393
Public
Repo size
1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin517 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024
- SHA
- 8087b828
Try a prompt
Model card
From the Hugging Face model README
DeciMamba Checkpoint (Baseline)
The official checkpoint of Mamba-130m, finetuned for Language Modeling over the PG-19 dataset as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
See our Github Repo for evalution and training scripts.
Bibtex:
@misc{benkish2024decimambaexploringlengthextrapolation,
title={DeciMamba: Exploring the Length Extrapolation Potential of Mamba},
author={Assaf Ben-Kish and Itamar Zimerman and Shady Abu-Hussein and Nadav Cohen and Amir Globerson and Lior Wolf and Raja Giryes},
year={2024},
eprint={2406.14528},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2406.14528},
}
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024