Downloads · 30 days
61
0% of all-time downloads
Mistral-7B-Merge-14-v0.5
EmbeddedLLM/Mistral-7B-Merge-14-v0.5
Mistral-7B-Merge-14-v0.5 is a text generation model from EmbeddedLLM. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Due to mlabonne/NeuralMarcoro14-7B updating its license to CC-BY-NC, our license will follow suit.
All-time downloads
16.4K
Public
Parameters
7.2B
14.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors14.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-4.0
- Model type
- mistral
- Access
- Public
- Created
- Jan 12, 2024
- Updated
- Jan 20, 2024
- SHA
- 1e526f1c
Try a prompt
Base models
Model card
From the Hugging Face model README
Update 2024-01-21
Due to mlabonne/NeuralMarcoro14-7B updating its license to CC-BY-NC, our license will follow suit.
Model Description
This is an experiment to test merging 14 models using DARE TIES 🦙
- We first merge 14 models to produce EmbeddedLLM/Mistral-7B-Merge-14-v0.3.
- The model is merged again using DARE TIES with:
Open LLM Leaderboard
| Average | 71.96 |
|---|---|
| ARC | 68.69 |
| HellaSwag | 86.45 |
| MMLU | 65.65 |
| TruthfulQA | 59.12 |
| Winogrande | 80.66 |
| GSM8K | 71.19 |
Chat Template
Either ChatML or Llama-2 chat template.
Merge Configuration
The merge config file for this model is here:
models:
- model: mistralai/Mistral-7B-v0.1
# no parameters necessary for base model
- model: EmbeddedLLM/Mistral-7B-Merge-14-v0.3
parameters:
weight: 0.3
density: 0.5
- model: Weyaxi/OpenHermes-2.5-neural-chat-v3-3-openchat-3.5-1210-Slerp
parameters:
weight: 0.2
density: 0.5
- model: openchat/openchat-3.5-0106
parameters:
weight: 0.2
density: 0.5
- model: mlabonne/NeuralMarcoro14-7B
parameters:
weight: 0.3
density: 0.5
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
int8_mask: true
tokenizer_source: union
dtype: bfloat16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- cc-by-nc-4.0
- Languages
- en
- Created
- Jan 12, 2024
- Updated
- Jan 20, 2024