Downloads · 30 days
1.5K
37% of all-time downloads
Zwielicht-27B
Nimbz/Zwielicht-27B
Zwielicht-27B is a machine learning model from Nimbz. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
<style .zw-shell { position: relative; padding: 10px; border-radius: 14px; background: linear-gradient(150deg, f0a15f22 0%, transparent 30%, transparent 70%, 8ea2ff22 100%), 05060d; box-shadow: 0 26px 80px rgba(0,0,0,…
All-time downloads
4K
Public
Parameters
27.8B
334 GB on disk
Likes
13
Public
Repo makeup
Click a slice to open those files.
.gguf223 GB · 80%
At a glance
- License
- apache-2.0
- Model type
- qwen3_5
- Access
- Public
- Created
- Sep 6, 2026
- Updated
- Sep 6, 2026
- SHA
- 9258b7af
Base models
Model card
From the Hugging Face model README
<style>
.zw-shell {
position: relative; padding: 10px; border-radius: 14px;
background: linear-gradient(150deg, #f0a15f22 0%, transparent 30%, transparent 70%, #8ea2ff22 100%), #05060d;
box-shadow: 0 26px 80px rgba(0,0,0,0.78);
}
.zw-main {
position: relative; overflow: hidden; border-radius: 10px; padding: 44px 40px 36px;
background-color: #04050b;
background-image:
radial-gradient(ellipse 80% 45% at 50% 106%, #f0a15f24 0%, transparent 62%),
radial-gradient(ellipse 90% 60% at 50% -12%, #8ea2ff20 0%, transparent 62%),
radial-gradient(ellipse 100% 70% at 50% 50%, rgba(255,255,255,0.016) 0%, transparent 72%);
color: #c2c8db;
font-family: 'Avenir Next','Segoe UI',Roboto,Helvetica,Arial,sans-serif;
line-height: 1.72;
}
.zw-main::before {
content: ""; position: absolute; inset: 0; pointer-events: none; z-index: 1;
background: linear-gradient(180deg, #8ea2ff12 0%, transparent 14%, transparent 84%, #f0a15f16 100%);
}
/* ---------- hero ---------- */
.zw-hero { text-align: center; position: relative; z-index: 5; }
.zw-title {
font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
font-size: 3.4rem; font-weight: 400; letter-spacing: 15px; margin: 30px 0 6px;
background: linear-gradient(96deg, #8ea2ff 0%, #dfe4f4 46%, #f0a15f 100%);
-webkit-background-clip: text; background-clip: text;
-webkit-text-fill-color: transparent; color: #eef1fa;
}
/* the horizon: the two lights meeting */
.zw-horizon { height: 1px; margin: 0 auto 20px; max-width: 460px; border: 0;
background: linear-gradient(90deg, transparent, #8ea2ff 22%, #e8ecf8 50%, #f0a15f 78%, transparent);
box-shadow: 0 0 22px #8ea2ff55, 0 0 22px #f0a15f44; }
.zw-badge {
display: inline-block; color: #a9b6ff; border: 1px solid #8ea2ff44;
background: #8ea2ff12; text-transform: uppercase; letter-spacing: 4px;
font-size: 0.7rem; font-weight: 600; padding: 7px 20px; border-radius: 999px;
}
.zw-tag { color: #7b8399; font-style: italic; font-size: 1.1rem; margin-top: 16px;
font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif; }
/* ---------- separators ---------- */
.zw-sep { display: flex; align-items: center; gap: 18px; margin: 46px 0 8px; position: relative; z-index: 5; }
.zw-sep.right { flex-direction: row-reverse; }
.zw-sep b {
font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
font-weight: 400; letter-spacing: 5px; font-size: 0.9rem; text-transform: uppercase;
color: #eef1fa; white-space: nowrap; padding: 6px 20px; border-radius: 999px;
background: rgba(10,14,26,0.82); border: 1px solid #1a2038;
}
.zw-sep b em { font-style: normal; color: #8ea2ff; margin-right: 9px; }
.zw-sep.right b em { color: #f0a15f; }
.zw-sep i { flex: 1; height: 1px; border: 0;
background: linear-gradient(90deg, transparent, #8ea2ff55 60%, #8ea2ff88); }
.zw-sep.right i { background: linear-gradient(90deg, #f0a15f88, #f0a15f55 40%, transparent); }
.zw-lead { font-size: 0.95rem; color: #7b8399; margin: 14px 2px 2px; position: relative; z-index: 5; }
/* ---------- chips ---------- */
.zw-chips { display: grid; grid-template-columns: repeat(auto-fit, minmax(148px,1fr)); gap: 12px; margin: 26px 0 4px; position: relative; z-index: 5; }
.zw-chip {
padding: 15px 12px; text-align: center; border-radius: 10px;
background: rgba(10,14,26,0.82); border: 1px solid #1a2038;
font-size: 0.76rem; letter-spacing: 2px; font-weight: 600; color: #eef1fa;
transition: transform .2s ease, box-shadow .2s ease, border-color .2s ease;
}
.zw-chip:nth-child(odd) { border-bottom: 2px solid #8ea2ff66; }
.zw-chip:nth-child(even) { border-bottom: 2px solid #f0a15f66; }
.zw-chip:hover { transform: translateY(-3px); border-color: #8ea2ff88; box-shadow: 0 10px 26px rgba(0,0,0,0.55); }
/* ---------- cards ---------- */
.zw-card {
position: relative; z-index: 5; margin: 22px 0; padding: 26px 30px;
background: linear-gradient(150deg, rgba(12,17,32,0.92), rgba(4,6,12,0.96));
border: 1px solid #1a2038; box-shadow: 0 12px 34px rgba(0,0,0,0.55);
}
.zw-card.L { border-left: 5px solid #8ea2ff; border-radius: 4px 20px 20px 4px; margin-right: 26px; }
.zw-card.R { border-right: 5px solid #f0a15f; border-radius: 20px 4px 4px 20px; margin-left: 26px; }
.zw-card.C { border-top: 3px solid #8ea2ff88; border-image: linear-gradient(90deg, #8ea2ff, #f0a15f) 1; border-radius: 0; }
.zw-card h3 {
font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
color: #eef1fa; margin: 0 0 12px; letter-spacing: 2px; font-weight: 400; font-size: 1.14rem;
}
.zw-card p { margin: 0 0 11px; font-size: 0.93rem; }
.zw-card p:last-child { margin-bottom: 0; }
.zw-rule { border: 0; height: 1px; margin: 15px 0;
background: linear-gradient(90deg, #8ea2ff55, #1a2038 55%, transparent); }
.zw-card.R .zw-rule { background: linear-gradient(90deg, transparent, #1a2038 45%, #f0a15f55); }
.zw-card code { background: #0b1020; border: 1px solid #1a2038; border-radius: 4px; padding: 1px 6px; font-size: 0.85em; color: #dfe4f4; }
/* phase head */
.zw-ph { display: flex; align-items: center; gap: 20px; margin-bottom: 14px; }
.zw-card.R .zw-ph { flex-direction: row-reverse; text-align: right; }
.zw-med {
flex: 0 0 76px; width: 76px; height: 76px; border-radius: 50%;
display: flex; align-items: center; justify-content: center;
font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
font-size: 1.7rem; color: #eef1fa;
background: radial-gradient(circle at 32% 26%, #8ea2ff33, transparent 62%), rgba(10,14,26,0.82);
border: 1px solid #8ea2ff66; box-shadow: 0 0 26px #8ea2ff33, inset 0 0 18px rgba(0,0,0,0.55);
}
.zw-card.R .zw-med { background: radial-gradient(circle at 32% 26%, #f0a15f33, transparent 62%), rgba(10,14,26,0.82);
border-color: #f0a15f66; box-shadow: 0 0 26px #f0a15f33, inset 0 0 18px rgba(0,0,0,0.55); }
.zw-ph h3 { margin: 0; }
.zw-ph small { display: block; color: #7b8399; letter-spacing: 3px; font-size: 0.66rem; text-transform: uppercase; margin-top: 3px; }
/* ---------- the depth ribbon: 64 layers, drawn ---------- */
.zw-ribbon { margin: 18px 0 6px; position: relative; z-index: 5; }
.zw-ribbon .bar { display: flex; gap: 2px; height: 34px; align-items: stretch; }
.zw-ribbon .bar i {
flex: 1 1 0; min-width: 0; border-radius: 2px; background: #0e1424; border-bottom: 2px solid transparent;
}
.zw-ribbon .bar i.a { border-bottom-color: #6e79a8; }
.zw-ribbon .bar i.c0 { background: #0b1020; }
.zw-ribbon .bar i.c2 { background: #8ea2ff4d; }
.zw-ribbon .bar i.c3 { background: #8ea2ffb3; }
.zw-ribbon .bar i.w1 { background: #f0a15f33; }
.zw-ribbon .bar i.w2 { background: #f0a15f4d; }
.zw-ribbon .bar i.w4 { background: #f0a15f; box-shadow: 0 0 12px #f0a15f66; }
.zw-ribbon .cap { display: flex; justify-content: space-between; font-size: 0.62rem; letter-spacing: 2px;
text-transform: uppercase; color: #6b7288; margin: 7px 2px 16px; }
.zw-ribbon .cap b { color: #aab3c9; font-weight: 600; letter-spacing: 2px; }
.zw-key { font-size: 0.72rem; color: #7b8399; letter-spacing: 1px; margin-top: 4px; }
.zw-key span { display: inline-block; width: 10px; height: 10px; border-radius: 2px; margin: 0 5px -1px 14px; }
.zw-key span:first-child { margin-left: 0; }
/* ---------- donor grid ---------- */
.zw-grid {
display: grid; grid-template-columns: minmax(0,0.9fr) minmax(0,1.1fr) minmax(0,1.5fr);
gap: 1px; background: #1a2038; border: 1px solid #1a2038; border-radius: 10px; overflow: hidden;
margin: 16px 0 4px; position: relative; z-index: 5; font-size: 0.87rem;
}
.zw-grid > div { background: #070b16; padding: 11px 14px; color: #aeb7c9; min-width: 0; overflow-wrap: anywhere; }
.zw-grid > .h { background: #0b1223; color: #8ea2ff; letter-spacing: 2px; font-size: 0.66rem; text-transform: uppercase; font-weight: 600; }
.zw-card.R .zw-grid > .h { color: #f0a15f; }
.zw-grid > div b { color: #eef1fa; }
/* ---------- misc ---------- */
.zw-btns { display: flex; gap: 12px; flex-wrap: wrap; margin: 14px 0 2px; position: relative; z-index: 5; }
.zw-btn { display: inline-block; padding: 10px 22px; border-radius: 999px; border: 1px solid #8ea2ff44; color: #a9b6ff !important; text-decoration: none !important; letter-spacing: 2px; font-size: 0.76rem; background: rgba(10,14,26,0.82); transition: transform .2s ease; }
.zw-btn:hover { transform: translateY(-2px); box-shadow: 0 8px 22px #8ea2ff33; }
.zw-btn.s { border-color: #f0a15f44; color: #f0a15f !important; }
.zw-kv { display: grid; grid-template-columns: minmax(0,1fr) minmax(0,1fr); gap: 1px; background: #1a2038; border: 1px solid #1a2038; border-radius: 10px; overflow: hidden; margin: 16px 0; position: relative; z-index: 5; }
.zw-kv > div { background: #070b16; padding: 10px 15px; font-size: 0.87rem; color: #aeb7c9; }
.zw-kv > .v { color: #eef1fa; font-weight: 600; }
.zw-kv > .h { background: #0b1223; color: #8ea2ff; letter-spacing: 2px; font-size: 0.66rem; text-transform: uppercase; }
.zw-det { margin: 4px 0 2px; }
.zw-det summary { cursor: pointer; color: #8ea2ff; letter-spacing: 2px; padding: 12px 0; font-size: 0.84rem; }
.zw-card.R .zw-det summary { color: #f0a15f; }
.zw-cred ul { margin: 0; padding-left: 18px; }
.zw-cred li { margin-bottom: 9px; font-size: 0.9rem; }
.zw-foot { text-align: center; letter-spacing: 14px; margin-top: 40px; opacity: 0.5; position: relative; z-index: 5;
background: linear-gradient(90deg, #8ea2ff, #f0a15f); -webkit-background-clip: text; background-clip: text; -webkit-text-fill-color: transparent; }
</style>
<div class="zw-shell">
<div class="zw-main">
<div class="zw-hero">
<video src="https://huggingface.co/Nimbz/Zwielicht-27B/resolve/main/zwielicht-27b-header.mp4" poster="https://huggingface.co/Nimbz/Zwielicht-27B/resolve/main/zwielicht-27b-header.png" width="100%" autoplay loop muted playsinline style="display:block;border-radius:10px;border:1px solid #8ea2ff44;box-shadow:0 0 70px #8ea2ff2e;margin:0 auto 4px;">
<img src="https://huggingface.co/Nimbz/Zwielicht-27B/resolve/main/zwielicht-27b-header.png" width="100%" style="display:block;border-radius:10px;border:1px solid #8ea2ff44;box-shadow:0 0 70px #8ea2ff2e;margin:0 auto 4px;">
</video>
<h1 class="zw-title">ZWIELICHT</h1>
<hr class="zw-horizon">
<div class="zw-badge">27B · Qwen 3.8 · Merge</div>
<div class="zw-tag">“Will the lights go out?”</div>
</div>
<div class="zw-chips">
<div class="zw-chip">FIVE MODELS</div><div class="zw-chip">TWO PHASES</div><div class="zw-chip">SURGICAL HERETIC</div><div class="zw-chip">EXPERIMENTAL</div><div class="zw-chip">THINKING</div><div class="zw-chip">ROLEPLAY MERGE</div>
</div>
<div class="zw-sep"><i></i><b><em>☀</em>Some Info</b><i></i></div>
<div class="zw-card C">
<h3>◆ Please use with reasoning on</h3>
<p>This model - in general - needs reasoning to be competent enough as a roleplay model. Benchmarks also prove that thinking with Qwen 3.8 improves roleplay score, creative writing, reduces the appearance of slop and improves lore recall. Use it without reasoning at your own risk.</p>
<hr class="zw-rule">
<p>I've changed the <code>reasoning_effort</code> from <code>xhigh</code> to <code>medium</code> and removed the <i>"prioritize correctness, consistency and clarity"</i> system injection. I think this improves the pacing of roleplay, but you can of course use your own jinja template or the stock one from the base model to your preference.</p>
</div>
<div class="zw-sep right"><i></i><b><em>⬛</em>GGUF quants</b><i></i></div>
<div class="zw-lead">Thank you to anyone providing quants!</div>
<div class="zw-btns">
<a class="zw-btn" href="https://huggingface.co/Nimbz/Zwielicht-27B/tree/main/static-gguf/up">STATIC</a>
<a class="zw-btn s" href="#">IMATRIX (SOON)</a>
</div>
<div class="zw-sep"><i></i><b><em>☰</em>The merge</b><i></i></div>
<div class="zw-lead">Two <b style="color:#eef1fa;">della_linear</b> passes over <b style="color:#eef1fa;">Qwen3.8-27B</b>, on the same <a href="https://github.com/zerofata/mergekit">zerofata’s mergekit</a>, but this time with a <code>qwen3_5</code> architecture definition that had to be brought up by my buddy Claude, as there seemed to be no 3.8 version of it.</div>
<div class="zw-card L"><div class="zw-ph"><div class="zw-med">I</div><div><h3>Comprehension</h3><small>Of The Residual Stream</small></div></div>
<div class="zw-grid"><div class="h">Model</div><div class="h">Purpose</div><div class="h">Scope</div>
<div><b>Dark-Scarlett v2.0</b></div><div>very good RP thinking, good response length</div><div>0.65 (L0-47) → 0.45 (L48-63)<br>density 0.85</div>
<div><b>Omega-Convergence</b></div><div>adherence, coherence</div><div>0.65 (L0-47) → 0.45 (L48-63)<br>density 0.85</div>
<div><b>Serenity-27B</b></div><div>ERP, pacing</div><div>0.45 → 0.30 · density 0.75</div>
<div><b>Heimdallr-27B</b></div><div>adherence, vocab & less slop</div><div>0.28 → 0.18 · density 0.70</div>
</div>
<div class="zw-ribbon">
<div class="bar"><i class="c3" title="L0"></i><i class="c3" title="L1"></i><i class="c3" title="L2"></i><i class="c3 a" title="L3"></i><i class="c3" title="L4"></i><i class="c3" title="L5"></i><i class="c3" title="L6"></i><i class="c3 a" title="L7"></i><i class="c3" title="L8"></i><i class="c3" title="L9"></i><i class="c3" title="L10"></i><i class="c3 a" title="L11"></i><i class="c3" title="L12"></i><i class="c3" title="L13"></i><i class="c3" title="L14"></i><i class="c3 a" title="L15"></i><i class="c3" title="L16"></i><i class="c3" title="L17"></i><i class="c3" title="L18"></i><i class="c3 a" title="L19"></i><i class="c3" title="L20"></i><i class="c3" title="L21"></i><i class="c3" title="L22"></i><i class="c3 a" title="L23"></i><i class="c3" title="L24"></i><i class="c3" title="L25"></i><i class="c3" title="L26"></i><i class="c3 a" title="L27"></i><i class="c3" title="L28"></i><i class="c3" title="L29"></i><i class="c3" title="L30"></i><i class="c3 a" title="L31"></i><i class="c3" title="L32"></i><i class="c3" title="L33"></i><i class="c3" title="L34"></i><i class="c3 a" title="L35"></i><i class="c3" title="L36"></i><i class="c3" title="L37"></i><i class="c3" title="L38"></i><i class="c3 a" title="L39"></i><i class="c3" title="L40"></i><i class="c3" title="L41"></i><i class="c3" title="L42"></i><i class="c3 a" title="L43"></i><i class="c3" title="L44"></i><i class="c3" title="L45"></i><i class="c3" title="L46"></i><i class="c3 a" title="L47"></i><i class="c2" title="L48"></i><i class="c2" title="L49"></i><i class="c2" title="L50"></i><i class="c2 a" title="L51"></i><i class="c2" title="L52"></i><i class="c2" title="L53"></i><i class="c2" title="L54"></i><i class="c2 a" title="L55"></i><i class="c2" title="L56"></i><i class="c2" title="L57"></i><i class="c2" title="L58"></i><i class="c2 a" title="L59"></i><i class="c2" title="L60"></i><i class="c2" title="L61"></i><i class="c2" title="L62"></i><i class="c2 a" title="L63"></i></div>
<div class="cap"><b>L0</b><span>Per Layer Impact · gate/up · q/k/v</span><b>L63</b></div>
</div>
<details class="zw-det"><summary>▸ Phase 1: della_linear</summary>
merge_method: della_linear
base_model: ./Qwen/Qwen3.8-27B
dtype: bfloat16
tokenizer_source: base
parameters:
normalize: false
epsilon: 0.05
lambda: 1.0
int8_mask: true
models:
- model: ./ReadyArt/Dark-Scarlett-27B-v2.0
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.85 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.0.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.1.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.2.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.3.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.4.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.5.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.6.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.7.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.8.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.9.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.10.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.11.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.12.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.13.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.14.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.15.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.16.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.17.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.18.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.19.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.20.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.21.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.22.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.23.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.24.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.25.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.26.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.27.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.28.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.29.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.30.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.31.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.32.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.33.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.34.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.35.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.36.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.37.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.38.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.39.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.40.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.41.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.42.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.43.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.44.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.45.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.46.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.47.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.48.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.48.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.49.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.49.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.50.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.50.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.51.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.51.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.52.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.52.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.53.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.53.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.54.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.54.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.55.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.55.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.56.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.56.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.57.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.57.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.58.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.58.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.59.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.59.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.60.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.60.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.61.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.61.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.62.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.62.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.63.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.63.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.v_proj, value: 0.45 }
- { value: 0.0 }
- model: ./ReadyArt/Omega-Convergence-27B-v1.0
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.85 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.0.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.1.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.2.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.3.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.3.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.4.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.5.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.6.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.7.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.7.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.8.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.9.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.10.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.11.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.11.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.12.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.13.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.14.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.15.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.15.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.16.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.17.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.18.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.19.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.19.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.20.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.21.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.22.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.23.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.23.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.24.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.25.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.26.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.27.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.28.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.29.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.30.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.31.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.32.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.33.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.34.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.35.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.35.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.36.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.37.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.38.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.39.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.40.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.41.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.42.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.43.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.44.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.45.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.46.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.65 }
- { filter: model.language_model.layers.47.mlp.up_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.65 }
- { filter: model.language_model.layers.47.self_attn.v_proj, value: 0.65 }
- { filter: model.language_model.layers.48.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.48.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.49.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.49.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.50.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.50.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.51.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.51.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.52.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.52.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.53.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.53.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.54.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.54.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.55.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.55.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.55.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.56.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.56.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.57.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.57.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.58.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.58.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.59.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.59.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.59.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.60.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.60.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.61.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.61.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.62.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.62.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.63.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.63.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.63.self_attn.v_proj, value: 0.45 }
- { value: 0.0 }
- model: ./ReadyArt/Serenity-27B
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.75 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.0.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.1.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.2.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.3.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.3.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.4.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.5.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.6.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.7.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.7.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.8.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.9.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.10.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.11.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.11.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.12.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.13.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.14.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.15.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.15.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.16.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.17.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.18.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.19.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.19.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.20.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.21.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.22.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.23.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.23.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.24.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.25.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.26.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.27.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.28.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.29.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.30.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.31.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.32.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.33.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.34.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.35.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.35.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.36.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.37.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.38.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.39.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.40.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.41.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.42.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.43.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.44.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.45.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.46.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.45 }
- { filter: model.language_model.layers.47.mlp.up_proj, value: 0.45 }
- { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.45 }
- { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.45 }
- { filter: model.language_model.layers.47.self_attn.v_proj, value: 0.45 }
- { filter: model.language_model.layers.48.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.48.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.49.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.49.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.50.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.50.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.51.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.51.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.3 }
- { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.3 }
- { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.3 }
- { filter: model.language_model.layers.52.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.52.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.53.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.53.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.54.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.54.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.55.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.55.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.3 }
- { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.3 }
- { filter: model.language_model.layers.55.self_attn.v_proj, value: 0.3 }
- { filter: model.language_model.layers.56.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.56.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.57.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.57.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.58.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.58.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.59.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.59.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.3 }
- { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.3 }
- { filter: model.language_model.layers.59.self_attn.v_proj, value: 0.3 }
- { filter: model.language_model.layers.60.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.60.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.61.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.61.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.62.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.62.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.63.mlp.gate_proj, value: 0.3 }
- { filter: model.language_model.layers.63.mlp.up_proj, value: 0.3 }
- { filter: model.language_model.layers.63.self_attn.q_proj, value: 0.3 }
- { filter: model.language_model.layers.63.self_attn.k_proj, value: 0.3 }
- { filter: model.language_model.layers.63.self_attn.v_proj, value: 0.3 }
- { value: 0.0 }
- model: ./ReadyArt/Heimdallr-27B
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.70 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.0.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.1.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.2.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.3.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.3.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.4.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.5.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.6.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.7.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.7.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.8.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.9.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.10.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.11.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.11.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.12.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.13.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.14.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.15.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.15.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.16.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.17.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.18.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.19.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.19.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.20.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.21.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.22.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.23.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.23.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.24.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.25.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.26.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.27.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.28.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.29.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.30.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.31.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.32.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.33.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.34.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.35.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.35.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.36.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.37.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.38.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.39.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.40.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.41.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.42.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.43.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.44.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.45.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.46.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.28 }
- { filter: model.language_model.layers.47.mlp.up_proj, value: 0.28 }
- { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.28 }
- { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.28 }
- { filter: model.language_model.layers.47.self_attn.v_proj, value: 0.28 }
- { filter: model.language_model.layers.48.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.48.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.49.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.49.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.50.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.50.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.51.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.51.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.18 }
- { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.18 }
- { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.18 }
- { filter: model.language_model.layers.52.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.52.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.53.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.53.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.54.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.54.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.55.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.55.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.18 }
- { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.18 }
- { filter: model.language_model.layers.55.self_attn.v_proj, value: 0.18 }
- { filter: model.language_model.layers.56.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.56.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.57.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.57.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.58.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.58.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.59.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.59.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.18 }
- { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.18 }
- { filter: model.language_model.layers.59.self_attn.v_proj, value: 0.18 }
- { filter: model.language_model.layers.60.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.60.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.61.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.61.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.62.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.62.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.63.mlp.gate_proj, value: 0.18 }
- { filter: model.language_model.layers.63.mlp.up_proj, value: 0.18 }
- { filter: model.language_model.layers.63.self_attn.q_proj, value: 0.18 }
- { filter: model.language_model.layers.63.self_attn.k_proj, value: 0.18 }
- { filter: model.language_model.layers.63.self_attn.v_proj, value: 0.18 }
- { value: 0.0 }
merge_method: della_linear
base_model: ./Zwielicht-27B-read
dtype: bfloat16
tokenizer_source: base
parameters:
normalize: false
epsilon: 0.05
lambda: 1.0
int8_mask: true
models:
- model: ./trohrbaugh/Qwen3.8-27B-heretic-ara
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 1.0 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.27.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.27.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.28.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.28.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.29.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.29.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.30.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.30.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.31.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.31.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.32.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.32.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.33.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.33.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.34.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.34.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.35.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.35.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.36.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.36.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.37.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.37.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.38.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.38.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.39.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.39.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.40.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.40.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.41.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.41.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.42.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.42.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.43.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.43.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.44.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.44.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.45.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.45.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.46.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.46.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.47.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.47.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.48.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.48.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.49.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.49.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.50.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.50.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.51.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.51.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.52.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.52.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.53.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.53.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.54.linear_attn.out_proj, value: 1.0 }
- { filter: model.language_model.layers.54.mlp.down_proj, value: 1.0 }
- { filter: model.language_model.layers.55.self_attn.o_proj, value: 1.0 }
- { filter: model.language_model.layers.55.mlp.down_proj, value: 1.0 }
- { value: 0.0 }
- model: ./ReadyArt/Dark-Scarlett-27B-v2.0
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.80 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.4.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.4.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.5.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.5.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.6.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.6.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.7.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.7.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.8.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.8.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.9.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.9.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.10.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.10.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.11.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.11.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.12.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.12.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.13.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.13.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.14.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.14.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.15.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.15.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.16.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.16.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.17.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.17.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.18.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.18.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.19.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.19.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.20.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.20.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.21.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.21.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.22.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.22.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.23.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.23.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.24.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.24.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.25.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.25.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.26.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.26.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.56.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.56.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.57.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.57.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.58.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.58.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.59.self_attn.o_proj, value: 0.3 }
- { filter: model.language_model.layers.59.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.60.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.60.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.61.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.61.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.62.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.62.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.63.self_attn.o_proj, value: 0.3 }
- { filter: model.language_model.layers.63.mlp.down_proj, value: 0.3 }
- { value: 0.0 }
- model: ./ReadyArt/Omega-Convergence-27B-v1.0
parameters:
density:
- { filter: A_log, value: 1.0 }
- { filter: dt_bias, value: 1.0 }
- { filter: conv1d, value: 1.0 }
- { filter: norm, value: 1.0 }
- { value: 0.80 }
weight:
- { filter: lm_head, value: 0.0 }
- { filter: model.language_model.embed_tokens, value: 0.0 }
- { filter: model.visual, value: 0.0 }
- { filter: mtp., value: 0.0 }
- { filter: model.language_model.layers.4.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.4.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.5.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.5.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.6.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.6.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.7.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.7.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.8.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.8.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.9.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.9.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.10.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.10.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.11.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.11.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.12.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.12.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.13.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.13.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.14.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.14.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.15.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.15.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.16.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.16.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.17.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.17.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.18.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.18.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.19.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.19.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.20.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.20.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.21.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.21.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.22.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.22.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.23.self_attn.o_proj, value: 0.45 }
- { filter: model.language_model.layers.23.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.24.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.24.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.25.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.25.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.26.linear_attn.out_proj, value: 0.45 }
- { filter: model.language_model.layers.26.mlp.down_proj, value: 0.45 }
- { filter: model.language_model.layers.56.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.56.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.57.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.57.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.58.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.58.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.59.self_attn.o_proj, value: 0.3 }
- { filter: model.language_model.layers.59.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.60.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.60.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.61.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.61.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.62.linear_attn.out_proj, value: 0.3 }
- { filter: model.language_model.layers.62.mlp.down_proj, value: 0.3 }
- { filter: model.language_model.layers.63.self_attn.o_proj, value: 0.3 }
- { filter: model.language_model.layers.63.mlp.down_proj, value: 0.3 }
- { value: 0.0 }
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Sep 6, 2026
- Updated
- Sep 6, 2026