Downloads · 30 days
80
0% of all-time downloads
mixtral-megamerge-dare-8x7b-v2
martyn/mixtral-megamerge-dare-8x7b-v2
mixtral-megamerge-dare-8x7b-v2 is a text generation model from martyn. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
The following models were merged with DARE using https://github.com/martyn/safetensors-merge-supermario
All-time downloads
40.8K
Public
Parameters
46.7B
93.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors93.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mixtral
- Access
- Public
- Created
- Dec 25, 2023
- Updated
- Dec 25, 2023
- SHA
- a2dda73a
Try a prompt
Model card
From the Hugging Face model README
mixtral megamerge 8x7b v2
The following models were merged with DARE using https://github.com/martyn/safetensors-merge-supermario
Mergelist
mistralai/Mixtral-8x7B-v0.1
mistralai/Mixtral-8x7B-Instruct-v0.1
cognitivecomputations/dolphin-2.6-mixtral-8x7b
Brillibitg/Instruct_Mixtral-8x7B-v0.1_Dolly15K
orangetin/OpenHermes-Mixtral-8x7B
NeverSleep/Noromaid-v0.1-mixtral-8x7b-v3
Merge command
python3 hf_merge.py to_merge_mixtral2.txt mixtral-2 -p 0.15 -lambda 1.95
Notes
- MoE gates were filtered for compatibility then averaged with
(tensor1 + tensor2)/2 - seems to generalize prompting formats and sampling settings
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- License
- apache-2.0
- Languages
- en
- Created
- Dec 25, 2023
- Updated
- Dec 25, 2023