Downloads · 30 days
15
18% of all-time downloads
NeuralHermes-Mistral-13b-DARE_blended-FAILURE
athirdpath/NeuralHermes-Mistral-13b-DARE_blended-FAILURE
NeuralHermes-Mistral-13b-DARE_blended-FAILURE is a text generation model from athirdpath. Use it when you need the model to write or continue text. It is set up for transformers.
Ooof, my man ain't feeling so hot, I'd pass on this one for now. Inverting and merging 20b Llama 2 models works quite well, evening out the gradients between slices. However, these 13b Mistrals seem to HATE it, I assu…
All-time downloads
82
Public
Parameters
13.3B
26.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors26.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mistral
- Access
- Public
- Created
- Dec 2, 2023
- Updated
- Dec 2, 2023
- SHA
- 5db200aa
Try a prompt
Model card
From the Hugging Face model README
Ooof, my man ain't feeling so hot, I'd pass on this one for now. Inverting and merging 20b Llama 2 models works quite well, evening out the gradients between slices. However, these 13b Mistrals seem to HATE it, I assume due to the unbalanced nature of my recipe. More study is required.
Recipe
merge_method: dare_ties
-
base_model: athirdpath/BigMistral-13b
-
model: athirdpath/NeuralHermes-Mistral-13b
weight: 0.60 / density: 0.35
-
model: athirdpath/NeuralHermes-Mistral-13b-INV
weight: 0.40 / density: 0.30
int8_mask: true
dtype: bfloat16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- Created
- Dec 2, 2023
- Updated
- Dec 2, 2023