Dark-Goetia-26B-A4B-LoRA-v2

Dark-Goetia-26B-A4B-LoRA-v2

Atmospheric genre LoRA adapter primarily for English-language roleplay (also works in Russian). Adds a darker, more literary tone to the narrative. Designed exclusively to adjust style and response structure in Dark Fantasy RP. The adapter contains no plots or characters from the training data.

This is the general-purpose release. Two experimental variants exist that split this adapter's target set in half — see Related adapters at the bottom.

Adapter versions

• goetia-book-lora-v2-main — 2 epochs → use the recommended scales below
• goetia-book-lora-v2-chk177 — 1 epoch → use 1.5× the scale from the table

Example: if you run main at 0.55, run chk177 at 1.1.

Recommended scale settings (main ver.)

Test setup: SillyTavern + Marinara's preset + 2000+ token character card.

The right ceiling depends on what your card asks for. Free-form prose RP tolerates much more than a card that demands structured output blocks (status trackers, World State sections, inner-thought blocks). Both regimes below are measured, not guessed.

Free-form prose RP — no strict formatting requirements

Performs reliably in all RP scenarios at scales above 0.5. The darker literary tone becomes most pronounced in Dark Fantasy settings. Best used in full roleplay sessions (character card + system prompt); isolated short prompts show a weaker effect.

Cards with strict structured output

If your card requires the model to emit formatted blocks every turn, the ceiling is considerably lower, and it differs by language:

Above these, the model keeps the style but starts silently dropping the formatted blocks. Russian is measurably more fragile than English at identical adapter strength — if you run bilingual sessions, tune to the Russian number.

Note on scaling: alpha/r = 2 is stored in the GGUF metadata, and llama.cpp multiplies by it on top of your scale. All numbers above assume a loader that respects that. If yours ignores the metadata, halve everything.

What's actually in the adapter

For anyone merging this or building on it — the trained delta isn't distributed evenly across attention. Frobenius norms of the effective delta ΔW = (B·A)·(alpha/r), aggregated over layers:

So roughly 61% OV / 39% QK by squared norm. That split turns out to matter: the OV half carries almost all of the style, while both halves independently contribute to formatting degradation — OV about 3× more readily per unit of weight change. The structured-output ceilings in the table above are set by the OV component alone.

Training

• Base: Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA (Gemma 4 MoE, abliterated)
• Method: QLoRA 4-bit, attention-only (q/k/v/o_proj), all 30 layers of the text tower, r=32, alpha=64, 2 epochs, completion-only loss
• Scope: 115 attention projections, not 120 — layers 5, 11, 17, 23 and 29 are global-attention layers with no v_proj at all. 22,978,560 trainable params (0.089%).
• Data: Novel broken down into scenes → scenes rewritten by a neural network into a synthetic RP dataset (actions and internal thoughts preserved, original author's prose style removed). Contains both English and Russian samples, English-primary.

Limitations

• 18+. Base model is abliterated, training data includes adult and dark content.
• Primarily optimized for English. Works in Russian, but the style effect is generally weaker and instruction-following is more fragile there — see the scale tables.
• Structured-output thresholds were measured with one generation per setting at a single context depth. Treat them as calibrated starting points, not hard constants.
• Inherits Gemma license terms.

Related adapters

v3 splits this adapter's 115 targets into two non-overlapping halves, trained separately on identical data with identical hyperparameters and seed:

• v3-A — v_proj + o_proj (55 modules). The style channel on its own, without the QK component. Experimental; useful if your card demands strict formatting and v2 keeps dropping it.
• v3-B — q_proj + k_proj (60 modules). Research artifact — weak style, published for reproducibility.

A + B = exactly this adapter's target set and parameter budget.