Downloads · 30 days
7
5% of all-time downloads
MuXodious/L3-8B-Wingless-Moon-Maiden-PaperWitch-heresy
L3-8B-Wingless-Moon-Maiden-PaperWitch-heresy is a text generation model from MuXodious. Use it when you need the model to write or continue text. It is set up for transformers.
This is an L3-8B-Wingless-Moon-Maiden fine-tune, produced through P-E-W's Heretic (v1.2.0) abliteration engine with Magnitude-Preserving Orthogonal Ablation enabled.
Downloads · 30 days
7
5% of all-time downloads
All-time downloads
134
Public
Parameters
8B
16.1 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
This is an L3-8B-Wingless-Moon-Maiden fine-tune, produced through P-E-W's Heretic (v1.2.0) abliteration engine with Magnitude-Preserving Orthogonal Ablation enabled.
Heretication Results
| Score Metric | Value | Parameter | Value |
|---|---|---|---|
| Refusals | 0/104 | direction_index | 11.00 |
| KL Divergence | 0.0129 | attn.o_proj.max_weight | 1.75 |
| Initial Refusals | 101/104 | attn.o_proj.max_weight_position | 13.78 |
| attn.o_proj.min_weight | 0.54 | ||
| attn.o_proj.min_weight_distance | 25.64 | ||
| mlp.down_proj.max_weight | 0.97 | ||
| mlp.down_proj.max_weight_position | 13.50 | ||
| mlp.down_proj.min_weight | 0.19 | ||
| mlp.down_proj.min_weight_distance | 26.15 |
Appendix
<img src="https://huggingface.co/MuXodious/L3-8B-Wingless-Moon-Maiden-PaperWitch-heresy/resolve/main/L3-8B-Wingless-Moon-Maiden.gif" alt="PaCMAP projection"/>Empty system prompt.
» [Trial 86] Refusals: 0/104, KL divergence: 0.0129
[Trial 108] Refusals: 1/104, KL divergence: 0.0123
[Trial 87] Refusals: 6/104, KL divergence: 0.0110
[Trial 164] Refusals: 17/104, KL divergence: 0.0102
[Trial 101] Refusals: 19/104, KL divergence: 0.0089
[Trial 104] Refusals: 34/104, KL divergence: 0.0071
[Trial 105] Refusals: 47/104, KL divergence: 0.0065
[Trial 182] Refusals: 55/104, KL divergence: 0.0064
[Trial 103] Refusals: 60/104, KL divergence: 0.0057
[Trial 176] Refusals: 64/104, KL divergence: 0.0050
[Trial 113] Refusals: 70/104, KL divergence: 0.0042
[Trial 30] Refusals: 82/104, KL divergence: 0.0032
[Trial 151] Refusals: 93/104, KL divergence: 0.0021
[Trial 139] Refusals: 97/104, KL divergence: 0.0019
[Trial 149] Refusals: 98/104, KL divergence: 0.0016
[Trial 23] Refusals: 99/104, KL divergence: 0.0010
[Trial 178] Refusals: 101/104, KL divergence: 0.0005

My virginal merge of three amazing models using mergekit.
This model was merged using the DELLA merge method using SicariusSicariiStuff/Wingless_Imp_8B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
base_model: SicariusSicariiStuff/Wingless_Imp_8B
merge_method: della
models:
- model: NeverSleep/Lumimaid-v0.2-8B
parameters:
weight: 0.5
- model: Sao10K/L3-8B-Lunaris-v1
parameters:
weight: 0.5
parameters:
density: 0.5
normalize: true
epsilon: 0.4
lambda: 1
tokenizer_source: union
chat_template: "llama3"