Downloads · 30 days
20
8% of all-time downloads
CleverMage-Mistral-13b-DARE_blended-FAILURE
athirdpath/CleverMage-Mistral-13b-DARE_blended-FAILURE
CleverMage-Mistral-13b-DARE_blended-FAILURE is a text generation model from athirdpath. Use it when you need the model to write or continue text. It is set up for transformers.
Oh no, he's dumb too! I have a working hypothesis. Inverting and merging 20b Llama 2 models works quite well, evening out the gradients between slices. However, these 13b Mistrals seem to HATE it, I assume due to the…
All-time downloads
252
Public
Parameters
13.3B
26.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors26.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mistral
- Access
- Public
- Created
- Dec 2, 2023
- Updated
- Dec 2, 2023
- SHA
- 8f7378e9
Try a prompt
Model card
From the Hugging Face model README
Oh no, he's dumb too! I have a working hypothesis. Inverting and merging 20b Llama 2 models works quite well, evening out the gradients between slices. However, these 13b Mistrals seem to HATE it, I assume due to the unbalanced nature of my recipe. More study is required.
Recipe
merge_method: dare_ties
-
base_model: athirdpath/BigMistral-13b
-
model: athirdpath/CleverMage-Mistral-13b
weight: 0.60 / density: 0.35
-
model: athirdpath/CleverMage-Mistral-13b-INV
weight: 0.40 / density: 0.30
int8_mask: true
dtype: bfloat16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- Created
- Dec 2, 2023
- Updated
- Dec 2, 2023