Downloads · 30 days
270
10% of all-time downloads
BlackMamba-2.8B
Zyphra/BlackMamba-2.8B
BlackMamba-2.8B is a machine learning model from Zyphra. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
<img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/JdxNtwFrmEAnjJ0MP5A3.jpeg" width="900" height="900" /
All-time downloads
2.8K
Public
Repo size
5.6 GB
Likes
31
Public
Repo makeup
Click a slice to open those files.
.bin5.6 GB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Feb 1, 2024
- Updated
- Feb 6, 2024
- SHA
- 521a7777
Spaces
Model card
From the Hugging Face model README
BlackMamba
<img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/JdxNtwFrmEAnjJ0_MP5A3.jpeg" width="900" height="900" /><img src="https://cdn-uploads.huggingface.co/production/uploads/65bc13717c6ad1994b6619e9/aHpEc5tnCJShO2Kn0f637.png" width="900" height="900" />BlackMamba: Mixture of Experts for State-space models
Quentin Anthony*, Yury Tokpanov*, Paolo Glorioso*, Beren Millidge*
Paper: https://arxiv.org/abs/2402.01771
About
We provide inference code for our BlackMamba model in our github repository: https://github.com/Zyphra/BlackMamba
BlackMamba is an novel architecture which combines state-space models (SSMs) with mixture of experts (MoE). It uses Mamba as its SSM block and switch transformer as its MoE block base. BlackMamba is extremely low latency for generation and inference, providing significant speedups over all of classical transformers, MoEs, and Mamba SSM models. Additionally, due to its SSM sequence mixer, BlackMamba retains linear compuational complexity in the sequence length.
- Library
- transformers
- License
- apache-2.0
- Created
- Feb 1, 2024
- Updated
- Feb 6, 2024