Downloads · 30 days
0
TinyCeNN-LM-Sharded-MoE-Top2
vtava/TinyCeNN-LM-Sharded-MoE-Top2
TinyCeNN-LM-Sharded-MoE-Top2 is a text generation model from vtava. Use it when you need the model to write or continue text. It is set up for transformers.
Transformer-free CeNN student using a parameter-neutral routed FFN.
Access
Public
Updated Sep 11, 2026
Repo size
967 KB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json3.6 MB · 79%
At a glance
- Task
- Text Generation
- Library
- transformers
- Access
- Public
- Created
- Sep 11, 2026
- Updated
- Sep 11, 2026
- SHA
- b7d632c7
Try a prompt
Base models
Model card
From the Hugging Face model README
TinyCeNN-LM Sharded MoE Top-2
Transformer-free CeNN student using a parameter-neutral routed FFN.
- 8 shards × 96 inner channels
- Top-2 routing
- trainable parameters: 481,729
- overhead vs plain CeNN: 0.320%
- held-out tokens: 65,536
- best CE: 5.115179
- teacher CE: 4.234912
- teacher-gap recovery: 84.55%
- Task
- Text Generation
- Library
- transformers
- Created
- Sep 11, 2026
- Updated
- Sep 11, 2026