Downloads · 30 days
14
4% of all-time downloads
Llama-3.1-MoE-8x8B-Instruct-raw
deutsche-telekom/Llama-3.1-MoE-8x8B-Instruct-raw
Llama-3.1-MoE-8x8B-Instruct-raw is a text generation model from deutsche-telekom. Use it when you need the model to write or continue text. The card lists the license as llama3.1.
All-time downloads
390
Public
Parameters
47.5B
95 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors95 GB · 100%
At a glance
- Task
- Text Generation
- License
- llama3.1
- Model type
- mixtral
- Access
- Public
- Created
- Aug 22, 2024
- Updated
- Aug 23, 2024
- SHA
- ada824bc
Try a prompt
Base models
Model card
From the Hugging Face model README
This model is "Built with Llama".
It is based on meta-llama/Meta-Llama-3.1-8B-Instruct and was created with the help of mergekit. This is the mergekit configuration we used: mergekit_moe_config.yml
It should be noted that this model is the raw model after merging. It still has randomly initialized router networks and will not be better than a single one of its expert models. This model requires further training before use.
This model has a total of 47.5B params, which is slightly more than the Mixtral 8x7b with its 46.7B params.
Licensing
This model is licensed under the Llama 3.1 Community License, Copyright (c) 2024 Philip May, Deutsche Telekom AG
Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
- Task
- Text Generation
- Type
- mixtral
- License
- llama3.1
- Languages
- en, de, fr, it, pt, hi
- Created
- Aug 22, 2024
- Updated
- Aug 23, 2024