Downloads · 30 days
15
1% of all-time downloads
decimamba-130m-niah
assafbk/decimamba-130m-niah
decimamba-130m-niah is a text generation model from assafbk. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
The official checkpoint of DeciMamba-130m, finetuned for the Passkey Retrieval task as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
All-time downloads
1.5K
Public
Repo size
1.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin671 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024
- SHA
- 73dcc80f
Try a prompt
Model card
From the Hugging Face model README
DeciMamba Checkpoint
The official checkpoint of DeciMamba-130m, finetuned for the Passkey Retrieval task as presented in DeciMamba: Exploring the Length Extrapolation Potential of Mamba.
See our Github Repo for evalution and training scripts.
Bibtex:
@misc{benkish2024decimambaexploringlengthextrapolation,
title={DeciMamba: Exploring the Length Extrapolation Potential of Mamba},
author={Assaf Ben-Kish and Itamar Zimerman and Shady Abu-Hussein and Nadav Cohen and Amir Globerson and Lior Wolf and Raja Giryes},
year={2024},
eprint={2406.14528},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2406.14528},
}
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Created
- Jul 5, 2024
- Updated
- Jul 5, 2024