rhymeai-gemma-4-gguf

RhymeAI Gemma 4 (E2B / E4B) — on-device lyric assistant models

LoRA fine-tunes of Gemma 4 E2B-it and E4B-it for RhymeAI / Writers Block, a native songwriting app. Trained to follow the app's exact on-device prompt format for four tasks: rhyme-constrained next-line suggestions, passage rewrites, single-word rewrites, and synonyms. Runs fully offline via llama.cpp.

Current versions

E2B v6 vs the previous E2B v2.1 on a fresh 60-song holdout (1,140 prompts × 3 seeds, blind pairwise judge, song-clustered 95% CIs): overall judge preference +16.9 pts [+9.2, +24.9], driven by passage rewrites (185 wins / 60 losses / 25 ties); next-line suggestions roughly level (net +2.2 pts [−6.2, +11.2]); single-word rewrites 1,046/1,080 vs 1,052/1,080; passage rewrites 534/540 vs 525/540; synonyms and six-line display equal.

The eval table below is the original v1 release.

Files

Eval (app's 43-case harness, temp 0.8 / top-k 40 / top-p 0.95, ctx 2048)

Training

QLoRA (r=16, alpha=16, 2 epochs, lr 2e-4 cosine, effective batch 16, prompt-masked loss) on 4,947 synthetic examples generated from original seed lyrics and machine-verified for rhyme + format correctness before training (CMU-dictionary rhyme checking). No copyrighted lyrics and no user data. Trained with Unsloth on a Colab L4; merged and quantized with llama.cpp (llama-export-lora + llama-quantize, Q4_K_M).

Prompt format: raw Gemma turn markers (<start_of_turn>user\n... / <start_of_turn>model\n), stop string <end_of_turn>.

Mac 12B v7 — rhyme options (2026-11-02)

A Mac Local AI GGUF option is available as gemma-4-12B-it-rhymeai-v7-Q4_K_M.gguf (7,381,382,976 bytes), based on google/gemma-4-12B-it. Model card and limitations · exact settings · SHA-256.

This is the existing v7 fine-tune with repeat penalty 1.1/lookback 64, not new v8 weights. The focused 48-case SIMPLE/VERSE test scored 93.75% clamped diagnostic option yield and 30/48 strict passes, with 9 excess-line responses versus E2B 0. Tested Mac mean latency was 2.110s versus E2B 0.553s. These are narrow app-allowlist/NEW_SOUND results, not a universal or human-quality claim. The 12B uses the raw Gemma4 envelope and one BOS described in its settings, rather than the legacy prompt example above. Existing E2B/E4B files remain unchanged.