spikyspace-attnlm-babylm10m

SpikySpace-AttnLM v3.2

SpikySpace-style spiking selective-scan causal language model for evaluation.

SpikySpace-AttnLM-v3_2 is a SpikySpace-inspired decoder-only causal language model. It adapts the event-gated state-space modeling idea from SpikySpace to BabyLM-scale language modeling, combining a spiking/event-sparse SSM backbone with partial local attention layers. The model should be viewed as a language-modeling adaptation inspired by SpikySpace, rather than a faithful reproduction of the original time-series forecasting architecture.

Inspiration and Citation

SpikySpace-AttnLM-v3_2 is inspired by the SpikySpace architecture. The citation below refers to the original SpikySpace paper, not to this model repository.

@misc{tang2026spikyspacespikingstatespace,
  title={SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting},
  author={Kaiwen Tang and Jiaqi Zheng and Yuze Jin and Yupeng Qiu and Guangda Sun and Zhanglu Yan and Weng-Fai Wong},
  year={2026},
  eprint={2601.02411},
  archivePrefix={arXiv},
  primaryClass={cs.NE},
  url={https://arxiv.org/abs/2601.02411}
}