embeddinggemma-300m-onnx

embeddinggemma-300m — full-pipeline ONNX export

A single-graph ONNX export of the complete sentence-transformers pipeline of google/embeddinggemma-300m: Transformer → Pooling → Dense(768→3072) → Dense(3072→768) → Normalize, all in one graph. Output sentence_embedding is the final L2-normalized 768-d sentence vector — no external pooling or projection needed (unlike transformer-only exports, which produce different vectors).

Built for Jarvis (a self-hosted assistant) so its memory/RAG runs on onnxruntime alone — no torch, no sentence-transformers at runtime.

Fidelity

Exported with torch.onnx.export (opset 17) from the official weights and verified against the torch pipeline: worst-case cosine similarity = 1.000000 across batch sizes and sequence lengths (short + long, batched + single). Vectors are interchangeable with the original model — existing indexes need no re-embedding.

Exporter + verification harness: src/scripts/export_embed_onnx.py

Files

model.onnx and model.onnx.data must sit in the same directory (ONNX external-data format).

Usage (torch-free)

import json, numpy as np, onnxruntime as ort
from tokenizers import Tokenizer

meta = json.load(open("meta.json"))
tok = Tokenizer.from_file("tokenizer.json")
tok.enable_truncation(max_length=meta["max_seq_length"])
tok.enable_padding(pad_id=meta["pad_token_id"], pad_token="<pad>")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

# embeddinggemma is ASYMMETRIC — use the matching prompt prefixes:
docs = ["title: none | text: The user's name is Ravi."]
enc = tok.encode_batch(docs)
vecs = sess.run(["sentence_embedding"], {
    "input_ids": np.array([e.ids for e in enc], dtype=np.int64),
    "attention_mask": np.array([e.attention_mask for e in enc], dtype=np.int64),
})[0]   # (batch, 768), already L2-normalized

Queries use the prefix task: search result | query: .

License

Gemma. This is a format conversion of Google's model; use is subject to the Gemma Terms of Use (bundled: GEMMA_TERMS_OF_USE.md, GEMMA_PROHIBITED_USE_POLICY.md, NOTICE) — by using these weights you agree to those terms.