RhymeAI-Gemma-4-E4B-v3-ONNX-WebGPU

RhymeAI Gemma 4 E4B v3 · ONNX WebGPU

This is a text-only Transformers.js/WebGPU export of the larger RhymeAI Gemma 4 E4B v3 lyric model. It is intended for experimental, fully local browser lyric generation.

Provenance and modifications

• Base model: google/gemma-4-E4B-it, via the pinned onnx-community/gemma-4-E4B-it-ONNX Q4F16 graph at revision 843f250f23bc91754def1e0f0db390dacd1e6b05.
• Fine-tune: gemma-4-E4B-it-rhymeai-v3-lora-adapter-F16.gguf from Shayde182/rhymeai-gemma-4-gguf at revision 27492c6696f2f868bbbe3aeafb633009ef932598.
• Modification: the rank-16 F16 LoRA was merged into 216 decoder matrices, and every affected matrix was requantized to ONNX Runtime MatMulNBits Q4F16. Only the text decoder and token embedding graph are included.
• Export size: 4,924,962,759 bytes.

The two resulting ONNX Runtime sessions were validated after conversion. Browser output quality still depends on the prompt and sampling settings.

Transformers.js

import {
  AutoTokenizer,
  Gemma4ForCausalLM,
} from "@huggingface/transformers";

const modelId = "emb1ter/RhymeAI-Gemma-4-E4B-v3-ONNX-WebGPU";

const [tokenizer, model] = await Promise.all([
  AutoTokenizer.from_pretrained(modelId),
  Gemma4ForCausalLM.from_pretrained(modelId, {
    device: "webgpu",
    dtype: "q4f16",
  }),
]);

Use a desktop Chromium browser with WebGPU and sufficient memory and browser storage. Model loading and generation should run in a Web Worker to avoid blocking the page.

Intended task

The upstream RhymeAI model was trained primarily for constrained songwriting operations such as rhyme suggestions and rewrites. This export can generate longer lyric drafts, but it was not specifically fine-tuned as a general long-form song generator.

License

Gemma 4 and the RhymeAI repository identify their models as Apache-2.0. This repository includes a copy of the Apache License 2.0. Modified files and the conversion performed here are described above.