Downloads · 30 days
36
3% of all-time downloads
Omega_Sapphira_Joyous-L3.3-70B-v1.0
cactopus/Omega_Sapphira_Joyous-L3.3-70B-v1.0
Omega_Sapphira_Joyous-L3.3-70B-v1.0 is a text generation model from cactopus. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.3.
<style .co-wrap { font-family: "Iowan Old Style", "Palatino Linotype", Palatino, Georgia, serif; max-width: 860px; margin: 0 auto; color: E8E4DA; background: 14161C; padding: 40px 44px 48px; border-radius: 4px; line-h…
All-time downloads
1.1K
Public
Parameters
70.6B
141 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors141 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3.3
- Model type
- llama
- Access
- Public
- Created
- Sep 3, 2026
- Updated
- Sep 4, 2026
- SHA
- 699c2415
Try a prompt
Base models
Model card
From the Hugging Face model README
<style>
.co-wrap {
font-family: "Iowan Old Style", "Palatino Linotype", Palatino, Georgia, serif;
max-width: 860px;
margin: 0 auto;
color: #E8E4DA;
background: #14161C;
padding: 40px 44px 48px;
border-radius: 4px;
line-height: 1.65;
}
.co-wrap h1, .co-wrap h2 { font-weight: 600; letter-spacing: -0.01em; }
.co-name {
font-size: 2.3em;
margin: 0 0 2px;
color: #F5F2EA;
line-height: 1.1;
}
.co-sub { color: #8E9AA8; font-size: 1.02em; margin: 0 0 30px; font-style: italic; }
.co-wrap h2 {
font-size: 1.22em;
color: #F5F2EA;
margin: 38px 0 12px;
padding-bottom: 7px;
border-bottom: 1px solid #2A2F39;
}
.co-wrap p, .co-wrap li { color: #C9C4B8; }
.co-h3 { font-size: 1.02em; color: #E8E4DA; font-weight: 600; margin: 24px 0 8px; }
.co-fig { margin: 4px 0 6px; }
.co-cap { color: #7E8794; font-size: 0.85em; margin: 0 0 8px; }
.co-key { display: flex; gap: 20px; flex-wrap: wrap; font-size: 0.88em; margin-bottom: 4px; }
.co-key span { display: flex; align-items: center; gap: 7px; color: #A9B2BD; }
.co-sw { width: 11px; height: 11px; border-radius: 2px; display: inline-block; }
.co-wrap table { border-collapse: collapse; width: 100%; font-size: 0.92em; margin: 6px 0 4px; }
.co-wrap th {
text-align: left; color: #8E9AA8; font-weight: 500;
border-bottom: 1px solid #2A2F39; padding: 7px 10px 7px 0;
}
.co-wrap td { padding: 7px 10px 7px 0; border-bottom: 1px solid #1F242C; color: #C9C4B8; }
.co-wrap td:first-child, .co-wrap th:first-child { color: #E8E4DA; }
.co-num { font-variant-numeric: tabular-nums; }
.co-wrap pre {
background: #0E1015; border: 1px solid #262B34; border-radius: 3px;
padding: 15px 17px; overflow-x: auto; font-size: 0.83em; line-height: 1.5;
}
.co-wrap code { font-family: "SF Mono", Menlo, Consolas, monospace; color: #9FBFB0; }
.co-wrap a { color: #8FA8DB; text-decoration: none; border-bottom: 1px solid #33415C; }
.co-wrap a:hover { border-bottom-color: #8FA8DB; }
.co-flag {
border-left: 2px solid #B4763A; padding: 2px 0 2px 16px; margin: 18px 0;
}
.co-flag p { color: #D6C4AC; margin: 0 0 8px; }
.co-foot { color: #6E7784; font-size: 0.85em; margin-top: 34px; }
/* cover art: shown whole at 2:3, bled into the page rather than framed */
.co-hero { position: relative; margin: 0 0 22px; }
.co-cover {
width: 400px; max-width: 88%; margin: 0 auto; position: relative;
aspect-ratio: 2 / 3;
}
.co-cover::before {
content: ""; position: absolute; inset: -12% -16% -9%;
background:
radial-gradient(ellipse 46% 40% at 28% 26%, rgba(214,150,68,0.17), transparent 68%),
radial-gradient(ellipse 58% 48% at 60% 62%, rgba(127,119,221,0.13), transparent 72%);
filter: blur(22px); z-index: 0;
}
.co-cover img {
position: relative; z-index: 1;
width: 100%; height: 100%; object-fit: contain; display: block;
border-radius: 47% 53% 44% 56% / 7% 7% 5% 5%;
-webkit-mask-image: radial-gradient(ellipse 84% 86% at 50% 44%, #000 62%, rgba(0,0,0,0.72) 84%, transparent 99%);
mask-image: radial-gradient(ellipse 84% 86% at 50% 44%, #000 62%, rgba(0,0,0,0.72) 84%, transparent 99%);
}
.co-cover::after {
content: ""; position: absolute; inset: 0; z-index: 2; pointer-events: none;
border-radius: 47% 53% 44% 56% / 7% 7% 5% 5%;
background:
linear-gradient(106deg, transparent 40%, rgba(226,232,245,0.05) 47%, transparent 53%),
radial-gradient(ellipse 80% 84% at 50% 44%, transparent 54%, rgba(20,22,28,0.62) 88%, rgba(20,22,28,0.98) 100%),
linear-gradient(to bottom, rgba(20,22,28,0.80) 0%, transparent 9%, transparent 74%, rgba(20,22,28,0.92) 98%);
}
/* pointer to the successor merge */
.co-next {
border: 1px solid #33415C; border-left: 3px solid #8FA8DB;
background: rgba(143,168,219,0.07);
border-radius: 3px; padding: 15px 20px; margin: 0 0 26px;
}
.co-next p { margin: 0; }
.co-next .co-next-label {
display: block; font-family: monospace; font-size: 0.74em;
letter-spacing: 0.15em; text-transform: uppercase;
color: #8FA8DB; margin-bottom: 6px;
}
.co-next a { font-weight: 600; }
@media (prefers-color-scheme: light) {
.co-wrap { background: #FBFAF7; color: #23282F; }
.co-name, .co-wrap h2, .co-h3 { color: #14171C; }
.co-wrap p, .co-wrap li, .co-wrap td { color: #3C434C; }
.co-wrap th, .co-sub, .co-cap { color: #6B7480; }
.co-wrap h2 { border-bottom-color: #DDD9D0; }
.co-wrap td { border-bottom-color: #EAE7E0; }
.co-wrap pre { background: #F2F0EA; border-color: #DDD9D0; }
.co-wrap code { color: #2C5F49; }
.co-wrap a { color: #3A5A9B; border-bottom-color: #B8C4DC; }
.co-cover::after {
background:
linear-gradient(106deg, transparent 40%, rgba(255,255,255,0.10) 47%, transparent 53%),
radial-gradient(ellipse 80% 84% at 50% 44%, transparent 54%, rgba(251,250,247,0.62) 88%, rgba(251,250,247,0.98) 100%),
linear-gradient(to bottom, rgba(251,250,247,0.80) 0%, transparent 9%, transparent 74%, rgba(251,250,247,0.92) 98%);
}
.co-next { border-color: #B8C4DC; border-left-color: #3A5A9B; background: rgba(58,90,155,0.06); }
.co-next .co-next-label { color: #3A5A9B; }
}
</style>
<div class="co-wrap">
<div class="co-hero">
<div class="co-cover">
<img src="cover.jpg" alt="">
</div>
</div>
<h1 class="co-name">Omega-Sapphira-Joyous-L3.3-70B-v1.0</h1>
<p class="co-sub">Three Llama 3.3 70B fine-tunes, blended by depth rather than by ratio.</p>
<div class="co-next">
<span class="co-next-label">Superseded</span>
<p><a href="https://huggingface.co/cactopus/Omega_Sapphira_Joyous-L3.3-70B-v1.1">Omega-Sapphira-Joyous-L3.3-70B-v1.1</a> is the follow-up to this merge, and is the one to start with. It performs the same graft with the Joyous share cut back sharply — roughly 43% less in the feed-forward blocks and 67% less in attention — because the amounts used here turned out to be too much. This page is kept for the record.</p>
</div>
<p><code>TLDR: Experimental. For a stable, well-behaveing merge use</code> <a href="https://huggingface.co/cactopus/Omega-Sapphira-L3.3-70B-v1.3">Omega-Sapphira-L3.3-70B-v1.3</a>. Most merges pick one number and apply it everywhere. This one applies a different number at every layer, and a different number again depending on whether the tensor is an attention block or a feed-forward block. The result is a model whose ancestry shifts as you move up the stack.</p>
<div class="co-fig">
<svg viewBox="0 0 720 260" xmlns="http://www.w3.org/2000/svg" style="width:100%;height:auto;">
<path d="M44.0,16.00 L52.2,16.00 L60.5,16.00 L68.8,16.00 L77.0,16.00 L85.2,16.00 L93.5,16.00 L101.8,16.00 L110.0,16.00 L118.2,16.00 L126.5,16.00 L134.8,16.00 L143.0,16.00 L151.2,16.00 L159.5,16.00 L167.8,16.00 L176.0,16.00 L184.2,16.00 L192.5,16.00 L200.8,16.00 L209.0,16.00 L217.2,16.00 L225.5,16.00 L233.8,16.00 L242.0,16.00 L250.2,16.00 L258.5,16.00 L266.8,16.00 L275.0,16.00 L283.2,16.00 L291.5,16.00 L299.8,16.00 L308.0,16.00 L316.2,16.00 L324.5,16.00 L332.8,16.00 L341.0,16.00 L349.2,16.00 L357.5,16.00 L365.8,16.00 L374.0,16.00 L382.2,16.00 L390.5,16.00 L398.8,16.00 L407.0,16.00 L415.2,16.00 L423.5,16.00 L431.8,16.00 L440.0,16.00 L448.2,16.00 L456.5,16.00 L464.8,16.00 L473.0,16.00 L481.2,16.00 L489.5,16.00 L497.8,16.00 L506.0,16.00 L514.2,16.00 L522.5,16.00 L530.8,16.00 L539.0,16.00 L547.2,16.00 L555.5,16.00 L563.8,16.00 L572.0,16.00 L580.2,16.00 L588.5,16.00 L596.8,16.00 L605.0,16.00 L613.2,16.00 L621.5,16.00 L629.8,16.00 L638.0,16.00 L646.2,16.00 L654.5,16.00 L662.8,16.00 L671.0,16.00 L679.2,16.00 L687.5,16.00 L695.8,16.00 L704.0,16.00 L704.0,182.65 L695.8,180.03 L687.5,177.42 L679.2,174.82 L671.0,172.24 L662.8,169.66 L654.5,167.10 L646.2,164.56 L638.0,162.02 L629.8,157.72 L621.5,153.45 L613.2,149.20 L605.0,144.99 L596.8,140.80 L588.5,136.65 L580.2,132.52 L572.0,128.42 L563.8,124.10 L555.5,119.83 L547.2,115.60 L539.0,111.41 L530.8,107.27 L522.5,103.17 L514.2,99.12 L506.0,95.11 L497.8,92.54 L489.5,90.00 L481.2,87.49 L473.0,85.02 L464.8,82.58 L456.5,80.17 L448.2,77.79 L440.0,75.44 L431.8,75.26 L423.5,75.08 L415.2,74.88 L407.0,74.68 L398.8,74.48 L390.5,74.27 L382.2,74.05 L374.0,73.82 L365.8,76.93 L357.5,80.05 L349.2,83.20 L341.0,86.36 L332.8,89.55 L324.5,92.76 L316.2,95.98 L308.0,99.23 L299.8,103.27 L291.5,107.37 L283.2,111.54 L275.0,115.76 L266.8,120.04 L258.5,124.38 L250.2,128.78 L242.0,133.24 L233.8,136.53 L225.5,139.86 L217.2,143.22 L209.0,146.61 L200.8,150.04 L192.5,153.51 L184.2,157.02 L176.0,160.56 L167.8,162.75 L159.5,164.95 L151.2,167.16 L143.0,169.39 L134.8,171.63 L126.5,173.88 L118.2,176.15 L110.0,178.43 L101.8,179.97 L93.5,181.53 L85.2,183.09 L77.0,184.65 L68.8,186.22 L60.5,187.80 L52.2,189.39 L44.0,190.98 Z" fill="#1D9E75" fill-opacity="0.80"/>
<path d="M44.0,190.98 L52.2,189.39 L60.5,187.80 L68.8,186.22 L77.0,184.65 L85.2,183.09 L93.5,181.53 L101.8,179.97 L110.0,178.43 L118.2,176.15 L126.5,173.88 L134.8,171.63 L143.0,169.39 L151.2,167.16 L159.5,164.95 L167.8,162.75 L176.0,160.56 L184.2,157.02 L192.5,153.51 L200.8,150.04 L209.0,146.61 L217.2,143.22 L225.5,139.86 L233.8,136.53 L242.0,133.24 L250.2,128.78 L258.5,124.38 L266.8,120.04 L275.0,115.76 L283.2,111.54 L291.5,107.37 L299.8,103.27 L308.0,99.23 L316.2,95.98 L324.5,92.76 L332.8,89.55 L341.0,86.36 L349.2,83.20 L357.5,80.05 L365.8,76.93 L374.0,73.82 L382.2,74.05 L390.5,74.27 L398.8,74.48 L407.0,74.68 L415.2,74.88 L423.5,75.08 L431.8,75.26 L440.0,75.44 L448.2,77.79 L456.5,80.17 L464.8,82.58 L473.0,85.02 L481.2,87.49 L489.5,90.00 L497.8,92.54 L506.0,95.11 L514.2,99.12 L522.5,103.17 L530.8,107.27 L539.0,111.41 L547.2,115.60 L555.5,119.83 L563.8,124.10 L572.0,128.42 L580.2,132.52 L588.5,136.65 L596.8,140.80 L605.0,144.99 L613.2,149.20 L621.5,153.45 L629.8,157.72 L638.0,162.02 L646.2,164.56 L654.5,167.10 L662.8,169.66 L671.0,172.24 L679.2,174.82 L687.5,177.42 L695.8,180.03 L704.0,182.65 L704.0,223.79 L695.8,223.21 L687.5,222.62 L679.2,222.03 L671.0,221.44 L662.8,220.85 L654.5,220.26 L646.2,219.67 L638.0,219.09 L629.8,218.31 L621.5,217.53 L613.2,216.76 L605.0,215.98 L596.8,215.21 L588.5,214.43 L580.2,213.66 L572.0,212.88 L563.8,211.70 L555.5,210.53 L547.2,209.35 L539.0,208.17 L530.8,207.00 L522.5,205.82 L514.2,204.64 L506.0,203.46 L497.8,201.89 L489.5,200.31 L481.2,198.73 L473.0,197.15 L464.8,195.57 L456.5,193.99 L448.2,192.42 L440.0,190.84 L431.8,189.29 L423.5,187.73 L415.2,186.18 L407.0,184.63 L398.8,183.08 L390.5,181.53 L382.2,179.98 L374.0,178.43 L365.8,179.01 L357.5,179.60 L349.2,180.19 L341.0,180.78 L332.8,181.37 L324.5,181.96 L316.2,182.55 L308.0,183.13 L299.8,184.69 L291.5,186.24 L283.2,187.79 L275.0,189.34 L266.8,190.89 L258.5,192.44 L250.2,193.99 L242.0,195.55 L233.8,196.91 L225.5,198.27 L217.2,199.64 L209.0,201.00 L200.8,202.37 L192.5,203.73 L184.2,205.10 L176.0,206.46 L167.8,207.26 L159.5,208.06 L151.2,208.87 L143.0,209.67 L134.8,210.47 L126.5,211.28 L118.2,212.08 L110.0,212.88 L101.8,213.47 L93.5,214.06 L85.2,214.65 L77.0,215.23 L68.8,215.82 L60.5,216.41 L52.2,217.00 L44.0,217.59 Z" fill="#C2410C" fill-opacity="0.80"/>
<path d="M44.0,217.59 L52.2,217.00 L60.5,216.41 L68.8,215.82 L77.0,215.23 L85.2,214.65 L93.5,214.06 L101.8,213.47 L110.0,212.88 L118.2,212.08 L126.5,211.28 L134.8,210.47 L143.0,209.67 L151.2,208.87 L159.5,208.06 L167.8,207.26 L176.0,206.46 L184.2,205.10 L192.5,203.73 L200.8,202.37 L209.0,201.00 L217.2,199.64 L225.5,198.27 L233.8,196.91 L242.0,195.55 L250.2,193.99 L258.5,192.44 L266.8,190.89 L275.0,189.34 L283.2,187.79 L291.5,186.24 L299.8,184.69 L308.0,183.13 L316.2,182.55 L324.5,181.96 L332.8,181.37 L341.0,180.78 L349.2,180.19 L357.5,179.60 L365.8,179.01 L374.0,178.43 L382.2,179.98 L390.5,181.53 L398.8,183.08 L407.0,184.63 L415.2,186.18 L423.5,187.73 L431.8,189.29 L440.0,190.84 L448.2,192.42 L456.5,193.99 L464.8,195.57 L473.0,197.15 L481.2,198.73 L489.5,200.31 L497.8,201.89 L506.0,203.46 L514.2,204.64 L522.5,205.82 L530.8,207.00 L539.0,208.17 L547.2,209.35 L555.5,210.53 L563.8,211.70 L572.0,212.88 L580.2,213.66 L588.5,214.43 L596.8,215.21 L605.0,215.98 L613.2,216.76 L621.5,217.53 L629.8,218.31 L638.0,219.09 L646.2,219.67 L654.5,220.26 L662.8,220.85 L671.0,221.44 L679.2,222.03 L687.5,222.62 L695.8,223.21 L704.0,223.79 L704.0,230.00 L695.8,230.00 L687.5,230.00 L679.2,230.00 L671.0,230.00 L662.8,230.00 L654.5,230.00 L646.2,230.00 L638.0,230.00 L629.8,230.00 L621.5,230.00 L613.2,230.00 L605.0,230.00 L596.8,230.00 L588.5,230.00 L580.2,230.00 L572.0,230.00 L563.8,230.00 L555.5,230.00 L547.2,230.00 L539.0,230.00 L530.8,230.00 L522.5,230.00 L514.2,230.00 L506.0,230.00 L497.8,230.00 L489.5,230.00 L481.2,230.00 L473.0,230.00 L464.8,230.00 L456.5,230.00 L448.2,230.00 L440.0,230.00 L431.8,230.00 L423.5,230.00 L415.2,230.00 L407.0,230.00 L398.8,230.00 L390.5,230.00 L382.2,230.00 L374.0,230.00 L365.8,230.00 L357.5,230.00 L349.2,230.00 L341.0,230.00 L332.8,230.00 L324.5,230.00 L316.2,230.00 L308.0,230.00 L299.8,230.00 L291.5,230.00 L283.2,230.00 L275.0,230.00 L266.8,230.00 L258.5,230.00 L250.2,230.00 L242.0,230.00 L233.8,230.00 L225.5,230.00 L217.2,230.00 L209.0,230.00 L200.8,230.00 L192.5,230.00 L184.2,230.00 L176.0,230.00 L167.8,230.00 L159.5,230.00 L151.2,230.00 L143.0,230.00 L134.8,230.00 L126.5,230.00 L118.2,230.00 L110.0,230.00 L101.8,230.00 L93.5,230.00 L85.2,230.00 L77.0,230.00 L68.8,230.00 L60.5,230.00 L52.2,230.00 L44.0,230.00 Z" fill="#7F77DD" fill-opacity="0.85"/>
<path d="M44.0,217.59 L52.2,217.00 L60.5,216.41 L68.8,215.82 L77.0,215.23 L85.2,214.65 L93.5,214.06 L101.8,213.47 L110.0,212.88 L118.2,212.08 L126.5,211.28 L134.8,210.47 L143.0,209.67 L151.2,208.87 L159.5,208.06 L167.8,207.26 L176.0,206.46 L184.2,205.10 L192.5,203.73 L200.8,202.37 L209.0,201.00 L217.2,199.64 L225.5,198.27 L233.8,196.91 L242.0,195.55 L250.2,193.99 L258.5,192.44 L266.8,190.89 L275.0,189.34 L283.2,187.79 L291.5,186.24 L299.8,184.69 L308.0,183.13 L316.2,182.55 L324.5,181.96 L332.8,181.37 L341.0,180.78 L349.2,180.19 L357.5,179.60 L365.8,179.01 L374.0,178.43 L382.2,179.98 L390.5,181.53 L398.8,183.08 L407.0,184.63 L415.2,186.18 L423.5,187.73 L431.8,189.29 L440.0,190.84 L448.2,192.42 L456.5,193.99 L464.8,195.57 L473.0,197.15 L481.2,198.73 L489.5,200.31 L497.8,201.89 L506.0,203.46 L514.2,204.64 L522.5,205.82 L530.8,207.00 L539.0,208.17 L547.2,209.35 L555.5,210.53 L563.8,211.70 L572.0,212.88 L580.2,213.66 L588.5,214.43 L596.8,215.21 L605.0,215.98 L613.2,216.76 L621.5,217.53 L629.8,218.31 L638.0,219.09 L646.2,219.67 L654.5,220.26 L662.8,220.85 L671.0,221.44 L679.2,222.03 L687.5,222.62 L695.8,223.21 L704.0,223.79" fill="none" stroke="#14161C" stroke-opacity="0.45" stroke-width="1"/>
<path d="M44.0,190.98 L52.2,189.39 L60.5,187.80 L68.8,186.22 L77.0,184.65 L85.2,183.09 L93.5,181.53 L101.8,179.97 L110.0,178.43 L118.2,176.15 L126.5,173.88 L134.8,171.63 L143.0,169.39 L151.2,167.16 L159.5,164.95 L167.8,162.75 L176.0,160.56 L184.2,157.02 L192.5,153.51 L200.8,150.04 L209.0,146.61 L217.2,143.22 L225.5,139.86 L233.8,136.53 L242.0,133.24 L250.2,128.78 L258.5,124.38 L266.8,120.04 L275.0,115.76 L283.2,111.54 L291.5,107.37 L299.8,103.27 L308.0,99.23 L316.2,95.98 L324.5,92.76 L332.8,89.55 L341.0,86.36 L349.2,83.20 L357.5,80.05 L365.8,76.93 L374.0,73.82 L382.2,74.05 L390.5,74.27 L398.8,74.48 L407.0,74.68 L415.2,74.88 L423.5,75.08 L431.8,75.26 L440.0,75.44 L448.2,77.79 L456.5,80.17 L464.8,82.58 L473.0,85.02 L481.2,87.49 L489.5,90.00 L497.8,92.54 L506.0,95.11 L514.2,99.12 L522.5,103.17 L530.8,107.27 L539.0,111.41 L547.2,115.60 L555.5,119.83 L563.8,124.10 L572.0,128.42 L580.2,132.52 L588.5,136.65 L596.8,140.80 L605.0,144.99 L613.2,149.20 L621.5,153.45 L629.8,157.72 L638.0,162.02 L646.2,164.56 L654.5,167.10 L662.8,169.66 L671.0,172.24 L679.2,174.82 L687.5,177.42 L695.8,180.03 L704.0,182.65" fill="none" stroke="#14161C" stroke-opacity="0.45" stroke-width="1"/>
<line x1="44" y1="230" x2="704" y2="230" stroke="#4A525E" stroke-width="1"/>
<line x1="44" y1="16" x2="44" y2="230" stroke="#4A525E" stroke-width="1"/>
<g font-family="monospace" font-size="10" fill="#7E8794">
<text x="38" y="234" text-anchor="end">0</text>
<text x="38" y="127" text-anchor="end">50%</text>
<text x="38" y="21" text-anchor="end">100%</text>
<text x="44" y="248">layer 0</text>
<text x="374" y="248" text-anchor="middle">40</text>
<text x="704" y="248" text-anchor="end">80</text>
</g>
</svg>
</div>
<div class="co-key">
<span><i class="co-sw" style="background:#7F77DD"></i>Joyous</span>
<span><i class="co-sw" style="background:#C2410C"></i>Sapphira</span>
<span><i class="co-sw" style="background:#1D9E75"></i>Omega</span>
</div>
<p class="co-cap">Ancestry of the feed-forward weights across the 80-layer stack. Shares are approximate; SLERP interpolates on a hypersphere, not linearly.</p>
<h2>What is actually in it</h2>
<p>Averaged over all 80 layers:</p>
<table>
<tr><th>Sub-block</th><th>Omega</th><th>Sapphira</th><th>Joyous</th></tr>
<tr><td>MLP — voice, style, knowledge</td><td class="co-num">52.9%</td><td class="co-num">34.2%</td><td class="co-num">12.9%</td></tr>
<tr><td>self_attn — structure, coherence</td><td class="co-num">68.1%</td><td class="co-num">24.1%</td><td class="co-num">7.9%</td></tr>
</table>
<p>Omega supplies structure and holds a majority of the attention weights at every depth. Sapphira supplies most of the prose, peaking at 53.9% of the feed-forward weights around layer 48. Joyous is a wash over the middle of the network, peaking at 24.1% around layer 40 and falling to near nothing at either end.</p>
<h2>How it was built</h2>
<p>Two SLERP passes. The first merged Sapphira into Omega with depth-graded curves, producing an intermediate. The second layered Joyous over that intermediate with a second, gentler set of curves.</p>
<p>Embeddings, per-layer norms, the final norm, and the output head are all taken unmodified from the first-pass merge, so nothing at the boundaries of the network is interpolated twice.</p>
<pre><code>slices:
- sources:
# t=0 => Omega-Sapphira v1.3, t=1 => Joyous
- model: Omega-Sapphira-L3.3-70B-v1.3
layer_range: [0, 80]
- model: allura-org/Llama-3.3-70B-Joyous
layer_range: [0, 80]
merge_method: slerp
base_model: Omega-Sapphira-L3.3-70B-v1.3
parameters:
t:
- filter: model.embed_tokens
value: 0.0
- filter: lm_head
value: 0.0
- filter: model.norm
value: 0.0
- filter: mlp
value: [0.058, 0.080, 0.110, 0.161, 0.219, 0.241, 0.183, 0.124, 0.080, 0.051, 0.029]
- filter: self_attn
value: [0.030, 0.038, 0.060, 0.090, 0.128, 0.158, 0.120, 0.083, 0.053, 0.030, 0.023]
- value: 0.0
dtype: bfloat16
tokenizer:
source: allura-org/Llama-3.3-70B-Joyous
chat_template: "llama3"</code></pre>
<p>The eleven values are control points at layers 0, 8, 16 … 80. mergekit interpolates linearly between them.</p>
<h2>How it behaves</h2>
<div class="co-flag">
<p>Based on my own testing: Joyous comes through more strongly than I expected from the numbers, and I find coherence a little softer than in the first-pass merge this was built on.</p>
<p>My reading is that the second pass travels further than the percentages imply — neither parent of that pass shares ancestry with Joyous, so SLERP covers a longer arc for a given <code>t</code>. If you want Omega-Sapphira's structural discipline intact, I would point you at <a href="https://huggingface.co/cactopus/Omega-Sapphira-L3.3-70B-v1.3">v1.3</a> instead. This one trades some of that for a warmer prose register, which is a tradeoff I like on some days and not others.</p>
</div>
<h2>Running it</h2>
<p>Llama 3 chat template. The tokenizer comes from Joyous rather than from the merge base — it carries the stock Llama 3.3 vocabulary along with <code>add_bos_token</code>, a <code>pad_token_id</code>, and a <code>generation_config.json</code>, none of which are present in Omega's tokenizer files.</p>
<p>40448 tokens of context fits on a 3090 + 4090 pair — 48GB of VRAM total — at EXL3 4.25bpw.</p>
<h3 class="co-h3">Samplers</h3>
<p>Two reference points. The left column is what Sapphira's author publishes for Sapphira itself; the right is what I actually run this merge at.</p>
<table>
<tr><th></th><th>Sapphira 0.2 (published)</th><th>This merge (in use)</th></tr>
<tr><td>Context</td><td class="co-num">—</td><td class="co-num">40448</td></tr>
<tr><td>Temperature</td><td class="co-num">0.95 – 1.1</td><td class="co-num">0.88</td></tr>
<tr><td>Min P</td><td class="co-num">0.015 – 0.03</td><td class="co-num">0.025</td></tr>
<tr><td>Top P</td><td class="co-num">0.97 – 1</td><td class="co-num">1</td></tr>
<tr><td>Typical P</td><td class="co-num">—</td><td class="co-num">1</td></tr>
<tr><td>TFS</td><td class="co-num">—</td><td class="co-num">1</td></tr>
<tr><td>Top K / Top A</td><td class="co-num">—</td><td class="co-num">0</td></tr>
<tr><td>XTC threshold</td><td class="co-num">0.11</td><td class="co-num">off</td></tr>
<tr><td>XTC probability</td><td class="co-num">0.15</td><td class="co-num">off</td></tr>
<tr><td>Repetition penalty</td><td class="co-num">—</td><td class="co-num">1.04</td></tr>
</table>
<p>The gap between the columns is mostly my preference. I run temperature below Sapphira's range, leave XTC off entirely, and let a light repetition penalty do that work instead. That is my response to the softening described above — tightening the distribution pulls back some of what the merge gave away. If you prefer Sapphira's looser settings, expect the softening to be more noticeable.</p>
<p>Treat my column as a starting point rather than a recommendation. Merged models sit between their parents in probability space, so it is worth re-tuning Min-P and the repetition penalty for your own taste rather than carrying anything across unchanged. In my experience a surprising amount of what reads as merge character is sampler character.</p>
<h2>Lineage</h2>
<p><a href="https://huggingface.co/ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1">ReadyArt/L3.3-The-Omega-Directive-70B-Unslop-v2.1</a> — a QLoRA fine-tune of Steelskull/L3.3-Shakudo-70b on a regex-filtered dataset of roughly 39M tokens, built to strip formulaic phrasing and to keep the model from writing on the user's behalf.</p>
<p><a href="https://huggingface.co/BruhzWater/Sapphira-L3.3-70b-0.2">BruhzWater/Sapphira-L3.3-70b-0.2</a> — a Multi-SLERP of Apocrypha 0.3 and Serpents-Tongue 0.3 over a cogito-v2-preview-llama-70B base. Serpents-Tongue itself draws on Anubis 70B v1.1, Fallen-Llama v1, Hanami-x1, mhnnn-x1, MythoNemo and Eden 0.3, so a good deal of the prose here has a long history behind it.</p>
<p><a href="https://huggingface.co/allura-org/Llama-3.3-70B-Joyous">allura-org/Llama-3.3-70B-Joyous</a> — brought in for its prose register. It is the most repetition-prone of the three, which is why its share is kept small and confined to the feed-forward blocks.</p>
<h2>Quantisations</h2>
<p>EXL3 and GGUF: pending.</p>
<h2>Before you use it</h2>
<p>All three parents are unaligned and intended for adult fiction. This merge inherits that and will engage with explicit and violent material without refusing. You are responsible for what you generate. You should be 18 or older.</p>
<p class="co-foot">Merged with <a href="https://github.com/arcee-ai/mergekit">mergekit</a> by Cactopus. Released under the Llama 3.3 Community License. All credit for the underlying work belongs to ReadyArt, BruhzWater, allura-org, and the many upstream authors listed on their cards.</p>
</div>
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3.3
- Languages
- en
- Created
- Sep 3, 2026
- Updated
- Sep 4, 2026