Downloads · 30 days
34
15% of all-time downloads
HGRN2-340M
linear-moe-hub/HGRN2-340M
HGRN2-340M is a machine learning model from linear-moe-hub. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Model of the paper MoM: Linear Sequence Modeling with Mixture-of-Memories and HGRN2: Gated Linear RNNs with State Expansion.
All-time downloads
229
Public
Parameters
374M
748 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors748 MB · 99%
At a glance
- License
- apache-2.0
- Model type
- hgrn2
- Access
- Public
- Created
- Mar 18, 2025
- Updated
- Mar 18, 2025
- SHA
- 0dbca617
Datasets
Model card
From the Hugging Face model README
Model of the paper MoM: Linear Sequence Modeling with Mixture-of-Memories and HGRN2: Gated Linear RNNs with State Expansion.
The model was trained on a sample of SlimPajama with 15B tokens.
Due to changes in the MLP layer structure in the latest version of fla, the weights cannot be loaded. You can use the version at fla instead.
- Type
- hgrn2
- License
- apache-2.0
- Languages
- en
- Created
- Mar 18, 2025
- Updated
- Mar 18, 2025