Downloads · 30 days
38
12% of all-time downloads
bigstral-12b-32k-8xMoE
bartowski/bigstral-12b-32k-8xMoE
bigstral-12b-32k-8xMoE is a text generation model from bartowski. Use it when you need the model to write or continue text. It is set up for transformers.
Made using mergekit MoE branch with the following config:
All-time downloads
317
Public
Parameters
81.5B
163 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors163 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mixtral
- Access
- Public
- Created
- Mar 7, 2024
- Updated
- Mar 7, 2024
- SHA
- 63df7c9f
Try a prompt
Base models
Model card
From the Hugging Face model README
bigstral-12b-32k-8xMoE
Made using mergekit MoE branch with the following config:
base_model: abacusai/bigstral-12b-32k
gate_mode: random
dtype: bfloat16
experts_per_token: 2
experts:
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- source_model: abacusai/bigstral-12b-32k
positive_prompts: []
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- Created
- Mar 7, 2024
- Updated
- Mar 7, 2024