Downloads · 30 days
7
8% of all-time downloads
sh0ck0r/Maginum-Cydoms-24B-heretic
Maginum-Cydoms-24B-heretic is a text generation model from sh0ck0r. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
7
8% of all-time downloads
All-time downloads
93
Public
Parameters
23.6B
47.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors47.1 GB · 100%
From the Hugging Face model README
| Parameter | Value |
|---|---|
| direction_index | 25.05 |
| attn.o_proj.max_weight | 1.47 |
| attn.o_proj.max_weight_position | 23.58 |
| attn.o_proj.min_weight | 0.76 |
| attn.o_proj.min_weight_distance | 14.12 |
| mlp.down_proj.max_weight | 1.12 |
| mlp.down_proj.max_weight_position | 29.91 |
| mlp.down_proj.min_weight | 0.35 |
| mlp.down_proj.min_weight_distance | 7.76 |
| Metric | This model | Original model (Casual-Autopsy/Maginum-Cydoms-24B) |
|---|---|---|
| KL divergence | 0.0072 | 0 (by definition) |
| Refusals | 14/100 | 61/100 |
This is a merge of pre-trained language models created using mergekit.
This model was merged using the following merge methods:
The following models were included in the merge:
The following YAML configurations were used to produce this model:
Maginum-Cydoms-S001:
models:
- model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- model: TheDrummer/Magidonia-24B-v4.3
parameters:
density: 1.0
weight: 1.0
- model: TheDrummer/Precog-24B-v1
parameters:
density: 0.4
weight: 0.6
- model: zerofata/MS3.2-PaintedFantasy-v3-24B
parameters:
density: 0.4
weight: 0.4
merge_method: ties
base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
parameters:
normalize: false
int8_mask: false
dtype: float32
tokenizer:
source: union
Maginum-Cydoms-S002:
models:
- model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- model: TheDrummer/Cydonia-24B-v4.3
parameters:
density: 1.0
weight: 1.0
epsilon: 0.0
- model: ReadyArt/4.2.0-Broken-Tutu-24b
parameters:
density: 0.4
weight: 0.6
epsilon: 0.2
- model: zerofata/MS3.2-PaintedFantasy-v2-24B
parameters:
density: 0.4
weight: 0.4
epsilon: 0.2
merge_method: della
base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
parameters:
normalize: false
int8_mask: false
dtype: float32
tokenizer:
source: union
models:
- model: Maginum-Cydoms-S001
- model: Maginum-Cydoms-S002
merge_method: slerp
base_model: Maginum-Cydoms-S001
parameters:
t:
- filter: self_attn
value: [0.3, 0.4, 0.6, 0.4, 0.3, 0.4, 0.6, 0.4, 0.3]
- filter: mlp
value: [0.7, 0.6, 0.4, 0.6, 0.7, 0.6, 0.4, 0.6, 0.7]
- value: 0.5
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union