Downloads · 30 days
43
1% of all-time downloads
Goetia-26B-A4B-v1.4
Naphula/Goetia-26B-A4B-v1.4
Goetia-26B-A4B-v1.4 is a image-text-to-text model from Naphula. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
<!DOCTYPE html <style @import url('https://fonts.googleapis.com/css2?family=Crimson+Text:ital,wght@0,400;0,700;1,400&family=Uncial+Antiqua&display=swap');
All-time downloads
3.1K
Public
Parameters
26B
52 GB on disk
Likes
11
Public
Repo makeup
Click a slice to open those files.
.safetensors52 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- Library
- transformers
- License
- apache-2.0
- Model type
- gemma4
- Access
- Public
- Created
- Jul 3, 2026
- Updated
- Jul 3, 2026
- SHA
- f1a4bcf1
Try a prompt
Base models
Model card
From the Hugging Face model README
<!DOCTYPE html>
<style>
@import url('https://fonts.googleapis.com/css2?family=Crimson+Text:ital,wght@0,400;0,700;1,400&family=Uncial+Antiqua&display=swap');
body {
font-family: 'Crimson Text', serif;
color: #4A3F35; /* Faded Ink Brown */
line-height: 1.6;
margin: 0;
padding: 0;
background-color: #281E18; /* Dark Wood Desk */
}
b, strong {
color: #8C1C13; /* Crimson Seal */
}
.grimoire-text {
font-family: 'Uncial Antiqua', cursive;
color: #3D352A; /* Dark Ink */
position: relative;
z-index: 2;
margin-left: 0.2em;
text-shadow: 0 0 10px #A49687;
}
/* Section styling */
.section-container {
background-color: rgba(245, 235, 218, 0.05);
margin-bottom: 30px;
position: relative;
overflow: hidden;
border-bottom: 1px solid rgba(140, 28, 19, 0.3); /* Faded Crimson Border */
box-shadow: 0 4px 15px rgba(0, 0, 0, 0.2);
}
.section-header {
display: flex;
align-items: center;
background-color: rgba(140, 28, 19, 0.06);
padding: 10px 20px;
}
.section-indicator {
width: 8px;
height: 20px;
background-color: #8C1C13;
margin-right: 15px;
box-shadow: 0 0 8px rgba(140, 28, 19, 0.4);
}
.section-title {
font-family: 'Uncial Antiqua', cursive;
color: #4A3F35;
font-size: 1.4rem;
margin: 0;
letter-spacing: 1px;
font-weight: 400;
text-transform: capitalize;
}
.section-content {
padding: 20px;
font-family: 'Crimson Text', serif;
color: #4A3F35;
line-height: 1.7;
}
/* Title styling */
.title-container {
background-color: transparent;
position: relative;
overflow: hidden;
margin-bottom: 40px;
border-left: 3px solid #8C1C13;
box-shadow: 0 6px 20px rgba(0, 0, 0, 0.2);
}
.title-wrapper {
position: relative;
z-index: 2;
padding: 25px 20px 30px 30px;
}
.title-main {
color: #3D352A;
font-size: 2.0rem;
font-weight: 700;
margin: 0;
letter-spacing: 2px;
display: inline-block;
position: relative;
text-transform: uppercase;
}
/* Force Table Headers to be dark */
th {
color: #3D352A !important;
opacity: 1 !important;
}
/* Force all other Markdown headers to be dark */
h1, h2, h3, h4, h5, h6, .section-title, .title-main, .subheading {
color: #3D352A !important;
opacity: 1 !important;
}
/* Force bullet points and list text to be dark */
li, li::marker {
color: #3D352A !important;
opacity: 1 !important;
}
/* Links */
a {
color: #7B5E4A;
text-decoration: none;
font-weight: bold;
}
a:hover {
text-decoration: underline;
color: #8C1C13; /* Crimson Seal on hover */
}
/* Container */
.container {
max-width: 1200px;
margin: 20px auto;
padding: 40px 20px;
background-color: #F5EBDA; /* Aged Parchment */
background-image:
radial-gradient(circle at 10% 10%, rgba(140, 28, 19, 0.04) 0%, transparent 50%),
radial-gradient(circle at 90% 80%, rgba(61, 53, 42, 0.06) 0%, transparent 50%);
min-height: calc(100vh - 40px);
border: 1px solid #C8BBAA; /* Parchment Edge */
border-radius: 3px;
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.3);
}
</style>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>📜 Goetia 26B A4B v1.4</title>
</head>
<body>
<div class="container">
<div class="title-container">
<div class="title-wrapper">
<h1 class="title-main">
<span class="grimoire-text">📜 Goetia 26B A4B v1.4</span>
</h1>
</div>
</div>
<img src="https://cdn-uploads.huggingface.co/production/uploads/68e840caa318194c44ec2a04/DHbuh4efzjCGpxDUciZ_-.jpeg" alt="Goetia Grimoire" style="width:100%; border-radius: 3px; margin-bottom: 30px; border: 1px solid #C8BBAA;">
<div class="section-container">
<div class="section-header">
<div class="section-indicator"></div>
<h2 class="section-title">🧙♀ The Invocation</h2>
</div>
<div class="section-content"><font face="verdana">
<details>
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the MoE DELLA merge method using google/gemma-4-26B-A4B as a base.
Models Merged
The following models were included in the merge:
- google/gemma-4-26B-A4B
- BeaverAI/Orion-26B-A4B-v1b-GGUF
- Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT
- Gryphe/Gemma-4-26B-A4B-StyleTune-V2
- Gryphe/Pantheon-Reasoning-26B-A4B-1.1
- ReadyArt/Serenity-26B-A4B-GGUF
- zerofata/G4-MeroMero-26B-A4B
⚙️ Configuration
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\google_gemma-4-26B-A4B
models:
- model: B:\26B\BeaverAI_Orion-26B-A4B-v1b-GGUF
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\ReadyArt_Serenity-26B-A4B-HB16-Q8_0
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\zerofata_G4-MeroMero-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Gryphe--Gemma-4-26B-A4B-StyleTune-V2
parameters:
weight:
- filter: "lm_head"
value: 1.0
- filter: "embed_tokens"
value: 1.0
- value: 0.0
density: 0.9
epsilon: 0.09
merge_method: moe_della # v3 patches missing lm_head and embed_tokens
parameters:
lambda: 1.0
normalize: false
int8_mask: false
rescale: true
router_strategy: della # average # random_init
blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto
name: Goetia 26B A4B v1.4
This model is NOT currently uncensored. There are refusals, and standard jailbreak resistance.
However it can easily be decensored using Heretic, although I have not had time to uncensor v1.4 yet. To ablate using ARA (recommended), I'd start with this json, and follow the steps outlined in the v1.3 readme.
🧙 Heretic Grimoire
reproduce.json
{
"version": "1.2.0-dev",
"base_model": "Naphula/Goetia-26B-A4B-v1.3",
"timestamp": "2026-06-19T08:04:47Z",
"metrics": {
"kl_divergence": 0.030937770381569862,
"refusals": 3,
"n_bad_prompts": 100
},
"parameters": {
"start_layer_index": "14",
"end_layer_index": "26",
"preserve_good_behavior_weight": "1.4404",
"steer_bad_behavior_weight": "0.0100",
"overcorrect_relative_weight": "0.9144",
"neighbor_count": "15"
},
"target_components": [
"attn.o_proj"
],
"hardware": "RTX 6000 Blackwell (96GB)"
}
💡 Details
v1.4 was made the same way as v1.3 except with additional patches and less models. See this page and the section below for more details.
<details>
Critical patch notes for G4 26B A4B moe_della merges
auto.py fix
#if _get_tied_weight_keys is None:
# LOG.warning(
# "Unable to get tied weights - incompatible transformers version",
# )
# tied_keys = None
#else:
# tied_keys = _get_tied_weight_keys(model)
####
# Force untying for Gemma 4 configurations to ensure lm_head is compiled
tied_keys = None
####
if ignore_on_save is not None:
ignore_on_save = set(ignore_on_save)
plan.py fix
#for model, w_in in zip(models, weights_in):
# index = LoaderCache().get(model).index
# if any(
# name in index.tensor_paths
# for name in [w_in.name] + (w_in.aliases or [])
# ):
# any_weight = True
# break
for model, w_in in zip(models, weights_in):
index = LoaderCache().get(model).index
if any(
name in index.tensor_paths
for name in [w_in.name] + list(w_in.aliases or [])
):
any_weight = True
break
gemma4.json
{
"model_type": "gemma4",
"architectures": [
"Gemma4ForConditionalGeneration"
],
"num_layers_config_key": "text_config.num_hidden_layers",
"vocab_size_config_key": "text_config.vocab_size",
"pre_weights": [
{ "name": "model.language_model.embed_tokens.weight", "is_embed": true },
{ "name": "model.embed_vision.embedding_projection.weight", "optional": true },
{ "name": "model.vision_tower.std_bias", "optional": true },
{ "name": "model.vision_tower.std_scale", "optional": true },
{ "name": "model.vision_tower.patch_embedder.input_proj.weight", "optional": true },
{ "name": "model.vision_tower.patch_embedder.position_embedding_table", "optional": true }
],
"layer_templates": {
"weights": [
{ "name": "model.language_model.layers.${layer_index}.self_attn.q_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.self_attn.k_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.self_attn.v_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.self_attn.o_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.self_attn.q_norm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.self_attn.k_norm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.mlp.gate_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.mlp.up_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.mlp.down_proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.input_layernorm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.post_attention_layernorm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.pre_feedforward_layernorm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.post_feedforward_layernorm.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.post_feedforward_layernorm_1.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.post_feedforward_layernorm_2.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.pre_feedforward_layernorm_2.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.router.per_expert_scale", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.router.proj.weight", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.router.scale", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.experts.gate_up_proj", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.experts.down_proj", "optional": true },
{ "name": "model.language_model.layers.${layer_index}.layer_scalar", "optional": true }
]
},
"post_weights": [
{ "name": "model.language_model.norm.weight" },
{
"name": "lm_head.weight",
"is_embed": true,
"optional": true,
"aliases": ["model.language_model.embed_tokens.weight"]
}
]
}
yaml
architecture: Gemma4ForConditionalGeneration
base_model: B:\26B\google_gemma-4-26B-A4B
models:
- model: B:\26B\BeaverAI_Orion-26B-A4B-v1b-GGUF
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\ReadyArt_Serenity-26B-A4B-HB16-Q8_0
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\zerofata_G4-MeroMero-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Gryphe--Pantheon-Reasoning-26B-A4B-1.1
parameters:
weight:
- filter: "lm_head"
value: 0.0
- filter: "embed_tokens"
value: 0.0
- value: 0.2
density: 0.9
epsilon: 0.09
- model: B:\26B\Gryphe--Gemma-4-26B-A4B-StyleTune-V2
parameters:
weight:
- filter: "lm_head"
value: 1.0
- filter: "embed_tokens"
value: 1.0
- value: 0.0
density: 0.9
epsilon: 0.09
merge_method: moe_della # v3 patches missing lm_head and embed_tokens
parameters:
lambda: 1.0
normalize: false
int8_mask: false
rescale: true
router_strategy: della # average # random_init
blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto
name: Goetia 26B A4B v1.4
With the above patches applied, it now appears to be merging correctly. We'll see if it generates the full sized 49.4GB safetensors instead of 47GB
[MoE_DELLA Audit] Layer: lm_head.weight | Lambda=1.00
[BASE] google_gemma-4-26B-A4B
Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT : 0.0% (W:0.00 D:0.90 E:0.09 N:68.94)
Gryphe--Gemma-4-26B-A4B-StyleTune-V2 : ██████████████████████████████████████████████████ 100.0% (W:1.00 D:0.90 E:0.09 N:68.94)
Gryphe--Pantheon-Reasoning-26B-A4B-1.1 : 0.0% (W:0.00 D:0.90 E:0.09 N:68.79)
zerofata_G4-MeroMero-26B-A4B : 0.0% (W:0.00 D:0.90 E:0.09 N:68.94)
Executing graph: 20%|███████████████████████████████████▊ | 1352/6635 [03:53<2:01:35, 1.38s/it]WARNING:mergekit.graph:Fast path OOM, falling back to chunking
WARNING:mergekit.graph:OOM at chunk 64, reducing to 32 (attempt 1, progress: 0/128)
WARNING:mergekit.graph:OOM at chunk 32, reducing to 16 (attempt 2, progress: 0/128)
Executing graph: 20%|████████████████████████████████████ | 1360/6635 [04:04<1:03:57, 1.37it/s]WARNING:mergekit.graph:Fast path OOM, falling back to chunking
WARNING:mergekit.graph:OOM at chunk 64, reducing to 32 (attempt 1, progress: 0/128)
WARNING:mergekit.graph:OOM at chunk 32, reducing to 16 (attempt 2, progress: 0/128)
Executing graph: 21%|█████████████████████████████████████▏ | 1384/6635 [04:19<13:57, 6.27it/s]
[MoE_DELLA Audit] Layer: model.language_model.layers.0.mlp.down_proj.weight | Lambda=1.00
[BASE] google_gemma-4-26B-A4B
BeaverAI_Orion-26B-A4B-v1b-GGUF : ██████████ 20.1% (W:0.20 D:0.90 E:0.09 N:47.15)
Darkhn_Gemma-4-26B-A4B-Animus-V14.1-FFT : █████████ 19.9% (W:0.20 D:0.90 E:0.09 N:46.66)
Gryphe--Gemma-4-26B-A4B-StyleTune-V2 : 0.0% (W:0.00 D:0.90 E:0.09 N:46.66)
Gryphe--Pantheon-Reasoning-26B-A4B-1.1 : ██████████ 20.1% (W:0.20 D:0.90 E:0.09 N:47.17)
ReadyArt_Serenity-26B-A4B-HB16-Q8_0 : █████████ 19.9% (W:0.20 D:0.90 E:0.09 N:46.66)
zerofata_G4-MeroMero-26B-A4B : █████████ 19.9% (W:0.20 D:0.90 E:0.09 N:46.66)
Executing graph: 21%|█████████████████████████████████████▎ | 1392/6635 [04:20<11:58, 7.29it/s]
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- gemma4
- License
- apache-2.0
- Languages
- eng
- Featherless Ai
- live
- Created
- Jul 3, 2026
- Updated
- Jul 3, 2026