Downloads · 30 days
9
16% of all-time downloads
medium_8192sl_gpu_64bs__softmax
gmongaras/medium_8192sl_gpu_64bs__softmax
medium_8192sl_gpu_64bs__softmax is a machine learning model from gmongaras. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Weights for the softmax model from the paper 2Mamba2Furious: Linear in Complexity, Competitive in Accuracy. This model variant just uses plain ol softmax (FlashAttention), used for the NIAH experiment. It was trained…
All-time downloads
56
Public
Parameters
735M
8.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt5.9 GB · 67%
At a glance
- Model type
- llama
- Access
- Public
- Created
- Feb 8, 2026
- Updated
- Feb 22, 2026
- SHA
- 0d2afade
Model card
From the Hugging Face model README
Weights for the softmax model from the paper 2Mamba2Furious: Linear in Complexity, Competitive in Accuracy.
This model variant just uses plain ol softmax (FlashAttention), used for the NIAH experiment. It was trained for 400K steps with a batch size of 32.
More details of the setup can be found in the Github repo.
Instructions on how to use this model can be found in https://github.com/gmongaras/2Mamba2Furious
- Type
- llama
- Created
- Feb 8, 2026
- Updated
- Feb 22, 2026