Downloads · 30 days
0
Qrzysztof/functiongemma-270m-it-prepaid-cards-v2-onnx
functiongemma-270m-it-prepaid-cards-v2-onnx is a text generation model from Qrzysztof. Use it when you need the model to write or continue text. It is set up for onnx. The card lists the license as gemma.
ONNX Runtime export of Qrzysztof/functiongemma-270m-it-prepaid-cards-v2 (a google/functiongemma-270m-it fine-tune for prepaid-card tool calling in 107 languages with noisy/multi-turn input) — for CPU, mobile, and web-…
Downloads · 30 days
0
Access
Public
Updated Aug 9, 2026
Repo size
1.6 GB
Likes
0
Public
Click a slice to open those files.
.data1.6 GB · 100%
From the Hugging Face model README
ONNX Runtime export of
Qrzysztof/functiongemma-270m-it-prepaid-cards-v2
(a google/functiongemma-270m-it fine-tune for prepaid-card tool calling in
107 languages with noisy/multi-turn input) — for CPU, mobile, and
web-browser inference (ONNX Runtime Web / Transformers.js).
| File | Size | Description |
|---|---|---|
model.onnx + model.onnx.data | 1.07 GB | fp32 export (exact reference) |
model-fp16.onnx + model-fp16.onnx.data | 536 MB | fp16 export — recommended for browsers |
Full-context decoder (no KV-cache inputs): every step feeds the whole context and returns full logits.
| Input | Shape | Dtype |
|---|---|---|
input_ids | [batch, seq] dynamic | int64 |
attention_mask | [batch, seq] dynamic | int64 |
position_ids | [batch, seq] dynamic | int64 |
logits (output) | [batch, seq, vocab] dynamic | fp32 / fp16 |
import numpy as np, onnxruntime as ort, torch
from transformers import AutoTokenizer
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
tokenizer = AutoTokenizer.from_pretrained("Qrzysztof/functiongemma-270m-it-prepaid-cards-v2")
prompt = tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True, tokenize=False)
ids = tokenizer(prompt, return_tensors="pt").input_ids
for _ in range(48):
logits = sess.run(None, {
"input_ids": ids.numpy().astype(np.int64),
"attention_mask": np.ones((1, ids.shape[1]), dtype=np.int64),
"position_ids": np.arange(ids.shape[1], dtype=np.int64).reshape(1, -1),
})[0]
ids = torch.cat([ids, torch.tensor([[logits[0, -1].argmax()]])], dim=1)
print(tokenizer.decode(ids[0], skip_special_tokens=False))
import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";
const tokenizer = await AutoTokenizer.from_pretrained("Qrzysztof/functiongemma-270m-it-prepaid-cards-v2-onnx");
const model = await AutoModelForCausalLM.from_pretrained("Qrzysztof/functiongemma-270m-it-prepaid-cards-v2-onnx",
{ dtype: "fp16", device: "wasm" });
The graph uses only standard ONNX ops (Gemm/MatMul/Add/Softmax/…), so it runs under the WASM backend. Full-context decode is O(seq²) — fine for a 270M model with short tool-call outputs.
Same as the parent model (see the SafeTensors card): synthetic data, uneven language quality, no backend. Performance note: fp32 CPU decode of this graph is slow (no KV cache); prefer fp16 in browsers and GGUF/MLX for interactive local use.
optimum-onnx is pinned to transformers <4.58 (incompatible with
transformers 5.x), so the export uses torch.onnx.export directly:
python3 export_onnx.py --model-dir <hf_model_dir> --out model.onnx # fp32
python3 export_onnx.py --model-dir <hf_model_dir> --out model-fp16.onnx --dtype fp16
Settings: eager attention, use_cache=False, opset 17, dynamic axes.
Correctness: verified token-identical greedy decoding vs the PyTorch
reference (export_onnx.py --check).
Same prompts & greedy decoding as the other formats, over a held-out v2 test subset (N=20, 32 max tokens — full-context CPU decode is slow).
| Format | Success rate |
|---|---|
| SafeTensors (reference) | 40/40 = 100% (N=40) |
| ONNX fp32 (ORT CPU) | 20/20 = 100% |
| ONNX fp16 (graph) | same graph semantics; tested identical logits in spot checks |
This model was fine-tuned with the tutorial below; you can use it as the starting point for a new tool set (or fine-tune google/functiongemma-270m-it directly).
A complete, minimal fine-tune of a FunctionGemma-class model on this data (follows the official FunctionGemma fine-tuning guide).
pip install torch transformers trl datasets accelerate
huggingface-cli login # accept the gemma license for google/functiongemma-270m-it
The Hub dataset stores messages/tools as JSON strings (Arrow cannot infer
the nested schema), and TRL's SFTTrainer needs a uniform struct schema, so
normalize first:
import json
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer
def normalize_messages(msgs):
out = []
for m in msgs:
n = {"role": m["role"], "content": m.get("content") or "", "name": None,
"tool_call_id": m.get("tool_call_id"), "tool_calls": None}
if m["role"] == "tool":
n["name"] = m["content"]["name"]
n["content"] = json.dumps(m["content"]["response"], ensure_ascii=False)
if m.get("tool_calls"):
n["tool_calls"] = [{"id": tc.get("id"), "type": tc.get("type", "function"),
"function": {"name": tc["function"]["name"],
"arguments": json.dumps(tc["function"]["arguments"], ensure_ascii=False)}}
for tc in m["tool_calls"]]
out.append(n)
return out
def rows_to_dataset(rows):
from datasets import Dataset
return Dataset.from_list([{
"messages": normalize_messages(r["messages"]),
"tools": json.dumps(r["tools"], ensure_ascii=False),
} for r in rows])
ds = load_dataset("Qrzysztof/ecommerce-chat-tool-calling", token=HF_TOKEN)["train"]
train_rows = [{"messages": json.loads(r["messages_json"]), "tools": json.loads(r["tools_json"])}
for r in ds if r["split"] == "train"]
train_ds = rows_to_dataset(train_rows)
import torch
from transformers import AutoModelForCausalLM
from trl import SFTConfig, SFTTrainer
model = AutoModelForCausalLM.from_pretrained("google/functiongemma-270m-it",
dtype=torch.bfloat16, attn_implementation="eager")
tokenizer = AutoTokenizer.from_pretrained("google/functiongemma-270m-it")
trainer = SFTTrainer(
model=model,
args=SFTConfig(
output_dir="functiongemma-ecommerce",
max_length=1024, # covers the longest sample + margin
packing=False, # keep tool calls intact (no cross-sample packing)
num_train_epochs=3,
per_device_train_batch_size=8,
learning_rate=5e-5,
lr_scheduler_type="constant",
warmup_steps=50,
bf16=True, # or fp16 on non-Ampere GPUs
eval_strategy="epoch",
report_to="none",
),
train_dataset=train_ds,
processing_class=tokenizer,
)
trainer.train()
TRL applies the FunctionGemma chat template with the per-sample tools
column; assistant_only_loss=True (default) masks everything but the model's
own turns, so it learns to emit tool calls — not to copy the schema.
ok = 0
for item in test_rows:
inputs = tokenizer.apply_chat_template(item["messages"][:-1], tools=item["tools"],
add_generation_prompt=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=256)
output = tokenizer.decode(out[0][len(inputs["input_ids"][0]):], skip_special_tokens=False)
expected = <expected tool name / args from expected_json>
ok += expected-tool-in-output and no-other-tool-in-output
trainer.push_to_hub("YOUR_USER/functiongemma-ecommerce")
Data
noise.py engine skips any token containing digits.template_id) and hold out
whole languages + (for the e-commerce set) whole schemas — that is the
only honest way to measure generalization.Training
packing=False for tool-calling data; packed sequences splice mid-call.max_length ≥ longest sample + a margin; ~1024 covers these datasets.Evaluation
Deployment
...-v2 (SafeTensors)...-tool-calling-v2