Downloads · 30 days
22
6% of all-time downloads
xxx777xxxASD/10.7B-Loyal-Mistral-Maid-32k-v0.2-A
10.7B-Loyal-Mistral-Maid-32k-v0.2-A is a text generation model from xxx777xxxASD. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Downloads · 30 days
22
6% of all-time downloads
All-time downloads
361
Public
Parameters
10.7B
21.5 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors21.5 GB · 100%
From the Hugging Face model README
![]()
Experimental merge, attempt to gain the roleplaying capabilities of Undi95/Toppy-M-7B and SanjiWatsuki/Loyal-Macaroni-Maid-7B while maintaining the context and capabilities of the original mistralai/Mistral-7B-Instruct-v0.2
The idea was that by combining two models with one self-merge, it would be possible to make each layer more unique, and therefore make the model “smarter” than a regular self-merge.
slices:
- sources:
- model: Mistral_Instruct_SelfMerge
layer_range: [0, 48]
- model: Loyal_Toppy_Maid
layer_range: [0, 48]
merge_method: slerp
base_model: Mistral_Instruct_SelfMerge
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5 # fallback for rest of tensors
dtype: bfloat16
slices:
- sources:
- model: Undi95/Toppy-M-7B
layer_range: [0, 24]
- sources:
- model: SanjiWatsuki/Loyal-Macaroni-Maid-7B
layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16
slices:
- sources:
- model: mistralai/Mistral-7B-Instruct-v0.2
layer_range: [0, 24]
- sources:
- model: mistralai/Mistral-7B-Instruct-v0.2
layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16