Downloads · 30 days
0
MambaBitPet-TinyStories
Maykeye/MambaBitPet-TinyStories
MambaBitPet-TinyStories is a machine learning model from Maykeye. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Access
Public
Updated Apr 28, 2024
Repo size
13.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin13.6 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Apr 28, 2024
- Updated
- Apr 28, 2024
- SHA
- 212cfdf1
Model card
From the Hugging Face model README
Mamba Bit!
Mamba with vocab size 2 bites again! This time we bite at tiny stories. I didn't bother preprocess them at all, during a training model took random char offset, converted it to bit string and fed to mamba. This time I didn't forget about residual connections nor about norm. As the result model was trained in BF16.
Training code included.
Example to run a model from CLI:
$ python mambabit.py "Run, kitten, run"
Run, kitten, running and jumping. She saw a big tree and thought it would be fun to share the tree. So, she went to the tree and started to climb the tree. She saw a big tree and thought it would be fun to share the tree. So, she went to the tree and saw a big red ball.
- License
- apache-2.0
- Created
- Apr 28, 2024
- Updated
- Apr 28, 2024