Mamba-Shedder-Mamba-2.2B

Mamba-Shedder Model: Mamba-Shedder-Mamba-2.2B

• Base Model: state-spaces/mamba-2.8b
• Pruned Components: 14 Mamba Blocks (Layer 2, 6, 12, 5, 10, 8, 13, 11, 26, 48, 19, 55, 15, 3)
• Recovery Tuning: No

Evaluation

git clone https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning.git
cd Mamba-Shedder
python eval.py --model_path <path to model>

Refer to our code repository for the environment information to run this command.

Ethical Considerations

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.

Model Sources

• Repository: https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder
• Paper: Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models

Citation

@inproceedings{munoz2025mambashedder,
  title = {Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models},
  author = {Mu{\~n}oz, J. Pablo and Yuan, Jinjie and Jain, Nilesh},
  booktitle = "Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025)",
    month = jun,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "",
}

Original Work Citation

This work builds upon work done by the State-Spaces team. Please see the following for additional citations of their work:

Repository: (state-spaces/mamba
Paper: Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality