Downloads · 30 days
61.1K
1% of all-time downloads
stories15M_MOE
ggml-org/stories15M_MOE
stories15M_MOE is a text generation model from ggml-org. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This model is ModelCloud/tinyllama-15M-stories repeated 4 times to make 4 experts.
All-time downloads
4.4M
Public
Parameters
36.4M
494 MB on disk
Likes
8
Public
Repo makeup
Click a slice to open those files.
.safetensors209 MB · 45%
Parameter types
How the weights are stored.
F1636.4M · 100%
Try a prompt
Model card
From the Hugging Face model README
stories15M_MOE
This model is ModelCloud/tinyllama-15M-stories repeated 4 times to make 4 experts.
The model is used for testing, not intended to be used in production (unless your product is some kind of bedtime story teller)
Weight of router is initialized randomly
shakespeare LoRA adapter
A LoRA adapter trained on first 100 paragraphs of shakespeare can be found inside moe_shakespeare15M
With input: Look in thy glass
- Original model generates:
Look in thy glass was a little girl. She was only three years old and she was three years old. She was - LoRA adapter generates:
Look in thy glass in love of the eye: That's when when the eye see thy on the sun'
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- mit
- Created
- Jul 11, 2024
- Updated
- Aug 5, 2024