Downloads · 30 days
20
5% of all-time downloads
decimamba-130m-pg19
assafbk/decimamba-130m-pg19
decimamba-130m-pg19 is a text generation model from assafbk. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
The official checkpoint of DeciMamba-130m, finetuned for Language Modeling over the PG-19 dataset as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
All-time downloads
416
Public
Repo size
1.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin671 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024
- SHA
- 750eb41a
Try a prompt
Model card
From the Hugging Face model README
DeciMamba Checkpoint
The official checkpoint of DeciMamba-130m, finetuned for Language Modeling over the PG-19 dataset as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
See our Github Repo for evalution and training scripts.
Bibtex:
@misc{benkish2024decimambaexploringlengthextrapolation,
title={DeciMamba: Exploring the Length Extrapolation Potential of Mamba},
author={Assaf Ben-Kish and Itamar Zimerman and Shady Abu-Hussein and Nadav Cohen and Amir Globerson and Lior Wolf and Raja Giryes},
year={2024},
eprint={2406.14528},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2406.14528},
}
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024