Downloads · 30 days
18
7% of all-time downloads
Llama-3-Teal-Instruct-2x8B-MoE
RDson/Llama-3-Teal-Instruct-2x8B-MoE
Llama-3-Teal-Instruct-2x8B-MoE is a text generation model from RDson. Use it when you need the model to write or continue text. It is set up for transformers.
<img src="https://i.imgur.com/eFrFD6v.jpeg" alt="drawing" width="640"/
All-time downloads
264
Public
Parameters
13.7B
27.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors27.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mixtral
- Access
- Public
- Created
- May 14, 2024
- Updated
- May 15, 2024
- SHA
- 55395ff8
Try a prompt
Model card
From the Hugging Face model README
<img src="https://i.imgur.com/eFrFD6v.jpeg" alt="drawing" width="640"/>
Llama-3-Teal-Instruct-2x8B-MoE
This is a experimental MoE created from meta-llama/Meta-Llama-3-8B-Instruct and nvidia/Llama3-ChatQA-1.5-8B using Mergekit.
Green + Blue = Teal.
Mergekit yaml file:
base_model: Meta-Llama-3-8B-Instruct
experts:
- source_model: Meta-Llama-3-8B-Instruct
positive_prompts:
- "explain"
- "chat"
- "assistant"
- source_model: Llama3-ChatQA-1.5-8B
positive_prompts:
- "python"
- "math"
- "solve"
- "code"
gate_mode: hidden
dtype: float16
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- Created
- May 14, 2024
- Updated
- May 15, 2024