Downloads · 30 days
18
25% of all-time downloads
tinymix-8x1b
mhenrichsen/tinymix-8x1b
tinymix-8x1b is a text generation model from mhenrichsen. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
71
Public
Parameters
6.4B
12.9 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors12.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Jan 2, 2024
- Updated
- Jan 2, 2024
- SHA
- 1fb48381
Try a prompt
Model card
From the Hugging Face model README
<div align="center">
TinyMix-8x1b
</div>This model is MoE consisting of 8 experts of TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
This model is untrained, and will likely perform worse than the dense version.
Will start training it very soon.
Idea by eastwind, who did it for the chat version of the model.
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Languages
- en
- Created
- Jan 2, 2024
- Updated
- Jan 2, 2024