Gemma-4-E4B-Nightcap

Gemma 4 E4B Nightcap

<small>One part Musica, a generous splash of TIES, and Bella behind the bar. Served after dark.</small>

A roleplay / creative-writing merge of three community fine-tunes of Gemma 4 E4B, mixed the boring way: measure the weight deltas first, pick parents that don't step on each other, build once, and blind-test the result against the strongest parent before deciding it's worth your bandwidth.

The goal was a single E4B that keeps Musica's prose and stability while picking up range it doesn't have on its own — grounded slice-of-life dialogue, comic timing, and a persona voice that will answer a direct question directly instead of deflecting into vibes. In blind A/B judging across 24 roleplay scenarios it trades wins evenly with Musica overall, with its stable wins exactly where the recipe aimed: grounded, blunt, character-forward scenes.

What's in the glass

Applied as weighted task arithmetic over google/gemma-4-E4B-it: base + 0.8·Δmusica + 0.225·Δties3 + 0.3·Δbella. The multimodal (audio/vision) towers are inherited from base effectively unchanged — all three parents are language-stack tunes.

<details> <summary>Recipe as mergekit-equivalent YAML</summary>

The build used independent tooling (it streams parents from the Hub and preserves every base tensor, including the phantom KV-shared-layer tensors some E4B re-saves drop), but the recipe is exactly this standard config:

merge_method: task_arithmetic
base_model: google/gemma-4-E4B-it
dtype: bfloat16
models:
  - model: AuriAetherwiing/G4-E4B-Musica-v1
    parameters:
      weight: 0.8
  - model: ToastyPigeon/Gemma4-E4B-TIES-3mask-r64
    parameters:
      weight: 0.225
  - model: juiceb0xc0de/bella-bartender-gemma-e4b
    parameters:
      weight: 0.3

</details>

Character notes

• Strong: single-character cards, grounded dialogue, persona consistency, comedy, answering the question you actually asked. Surprisingly good long-session coherence for an E4B.
• Trade-off: Musica alone keeps a slight edge in pure lyrical atmosphere and literary continuation. If that's your whole use case, Musica is right there.
• Weak: multiple distinct characters in one card (an ensemble sharing one system prompt). Voices can blend. This appears to be an E4B-class limit — the parents do it too — but it's this model's clearest weak spot.

Usage

• Format: standard Gemma 4 chat template, nothing custom. Works with the thinking channel on or off; most testing was done without.
• Samplers: Musica's recommendations carry over — temp 1.0, min-p 0.02, nSigma 2, no repetition penalties. Temp 0.8 / top-p 0.95 also tested fine.
• Quantization: Q8_0 recommended. Bella's author found her voice audibly sanded down below Q8, and part of that voice is what you're here for.

Credits

• AuriAetherwiing and ArliAI for Musica — the backbone of this merge and a genuinely lovely tune.
• ToastyPigeon for the TIES-3mask merge and the instruct-subspace masking technique behind it — the reason this ingredient adds prose without subtracting instruction-following.
• juiceb0xc0de for Bella — 9,300 pairs of one real human voice, and it shows.
• Google DeepMind for Gemma 4 E4B.

<details> <summary>Nerd details — how the parents and weights were chosen</summary>

The premise: most merge quality is decided before anything is built. So instead of a merge→test→re-merge loop, the pool was measured first and exactly one candidate recipe was built and gated.

Scouting the pool

Every E4B derivative on the Hub (~1,100 repos at the time) was swept down to ten genuinely distinct creative-writing/RP fine-tunes — dropping quant/format re-uploads, code and domain tunes, and uncensored-only edits, then reading the cards of what remained.

Measure, then select

Per-tensor weight deltas (Δ = fine-tune − base) were extracted for all ten, over the language stack only (audio/vision towers excluded from all statistics): exact per-tensor delta norms plus a deterministic subsample of coordinates, sampled at identical positions in every model so the candidates can be compared directly.

Measurement did the pruning:

• One popular "family" of tunes was substantially the same vector uploaded twice (pairwise cosine 0.54–0.58; one member was recoverable as a weighted sum of two others) — redundant, dropped.
• Another candidate's headline delta norm was ~99% quantization noise in the per-layer embedding table, not trained signal — dropped.
• A pair of DPO variants moved the weights so little (‖Δ‖ ≈ 0.5, ~40× quieter than the pool median) that they couldn't survive at any reasonable merge weight — dropped.

What survived selection was a lead voice (musica — the strongest all-round RP tune in the pool) plus two complements chosen because each brings a direction the lead measurably lacks, and all three are near-orthogonal in delta space — they learned different things, so their contributions add instead of fighting:

Delta magnitudes (language stack ℓ₂): ties3 26.6, musica 7.5, bella 1.7 — i.e. bella is a light seasoning by construction, not just by weight.

Choosing the weights

Candidate weight vectors were swept on the artifacts alone — no builds — against the trained-envelope statistic (below). The recipe kept the lead near full strength (0.8, preserving musica's voice and stability), scaled ties3 down to 0.225 (its raw delta is 3.5× the lead's; at higher weight it dominates the mix), and added bella at 0.3 — deliberately below her full strength, because a single-human-voice tune is a garnish that becomes an identity at high weight. Each parent's documented quirks cap its weight: dilution does not reliably remove a known failure mode, as the four-parent variant below demonstrated.

Constrain the build

Two guardrails picked the weights:

1. Trained envelope: at the chosen weights, almost no coordinate of the merged delta exceeds the largest movement any single parent trained at that coordinate (violation rate ≈ 0.7% after flooring out bf16 rounding noise, which is within the "clean" band calibrated on known-good and known-broken merges).
2. Weight-sum budget: Σw = 1.325. Empirically (on this family and a 12B sibling project), merges hold together up to roughly Σw ≈ 1.3 and degrade beyond it regardless of per-coordinate statistics — a heavier four-parent variant of this same recipe (Σw = 1.87) passed the envelope check and still lost a blind A/B against both Musica and this model, 3–9 and 2–10, reproducing its parents' documented failure modes at merge scale. This recipe is the one that survived.

As a free integrity check, regressing the built model's delta against the parent subsamples recovers the recipe weights (0.80 / 0.21 / 0.30) — confirming nothing unexpected leaked into the build.

Blind gate

Two independent rounds of 12 roleplay/creative scenarios (fresh seeds per round, identical samplers, per-prompt shuffled A/B, judged blind at Q8_0): 11–13 vs Musica — statistical parity, with stable wins split by territory: this merge on grounded/comedic/blunt scenes, Musica on atmosphere and literary continuation. Per the one-build philosophy, parity against the strongest parent plus added range = ship it, don't start a weight-tuning loop.

</details>