Downloads · 30 days
22
6% of all-time downloads
Qrzysztof/functiongemma-270m-it-prepaid-cards-v2-gguf
functiongemma-270m-it-prepaid-cards-v2-gguf is a text generation model from Qrzysztof. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as gemma.
GGUF conversion of Qrzysztof/functiongemma-270m-it-prepaid-cards-v2 (SafeTensors), a google/functiongemma-270m-it fine-tune that buys prepaid cards, checks balances, and lists transactions via tool calls in 107 langua…
Downloads · 30 days
22
6% of all-time downloads
All-time downloads
394
Public
Repo size
851 MB
Likes
0
Public
Click a slice to open those files.
.gguf851 MB · 100%
From the Hugging Face model README
GGUF conversion of Qrzysztof/functiongemma-270m-it-prepaid-cards-v2
(SafeTensors), a google/functiongemma-270m-it fine-tune that buys prepaid
cards, checks balances, and lists transactions via tool calls in 107 languages
with realistic noisy/multi-turn input.
Runnable with llama.cpp and any GGUF-compatible runtime (llama-cpp-python, LM Studio, Ollama, …).
| File | Size | Description |
|---|---|---|
model-f16.gguf | 551 MB | Full-precision weights |
model-Q8_0.gguf | 300 MB | 8-bit quantization — low-end hardware (see below) |
# CLI (llama.cpp)
llama-cli -m model-Q8_0.gguf -cnv -p "I want to buy a $20 card"
# llama-cpp-python — feed the FunctionGemma-rendered prompt
from llama_cpp import Llama
llm = Llama(model_path="model-Q8_0.gguf", n_ctx=4096, n_gpu_layers=0, verbose=False)
prompt = "<bos><start_of_turn>developer\nYou are a model that can do function calling with the following functions<start_function_declaration>..." # tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True)
print(llm(prompt, max_tokens=96, temperature=0.0)["choices"][0]["text"])
# <start_function_call>call:purchase_card{"amount": 20, "card_type": "digital_prepaid_visa", ...}<end_function_call>...
Note: llama-cli (master) had a REPL quirk when passing prompts with -p/-f
at the time of writing; llama-cpp-python (same backend) is the tested path.
Same as the parent model (see the SafeTensors card): synthetic data, uneven language quality, no backend — plus the usual quantization caveats below.
# 1. conversion (llama.cpp convert_hf_to_gguf.py; needs the llama.cpp repo layout)
python3 convert_hf_to_gguf.py <hf_model_dir> --outfile model-f16.gguf --outtype f16
# 2. 8-bit quantization
llama-quantize model-f16.gguf model-Q8_0.gguf Q8_0
Gemma3 270M gotcha: the HF tokenizer reports 262,146 tokens (two
model-specific vision tokens added at load). convert_hf_to_gguf.py asserts
max(vocab) < vocab_size and fails. Fix: in a conversion copy, drop the
*_token / model_specific_special_tokens keys from tokenizer_config.json
and prune added tokens with id ≥ 262144 from tokenizer.json → exactly
262,144 tokens, identical text behavior.
Same prompts & greedy decoding as the other formats, over the held-out v2 test subset (N=40).
| Format | Success rate |
|---|---|
| SafeTensors (reference) | 40/40 = 100% |
| GGUF f16 | TODO |
| GGUF Q8_0 | 40/40 = 100% |
Q8_0 shows no measurable drop on tool-name selection in this subset; arg wording can differ slightly from f16 in rare cases.
llama-quantize model-f16.gguf model-Q4_K_M.gguf Q4_K_M
(≈170 MB) trades a bit more accuracy.This model was fine-tuned with the tutorial below; you can use it as the starting point for a new tool set (or fine-tune google/functiongemma-270m-it directly).
A complete, minimal fine-tune of a FunctionGemma-class model on this data (follows the official FunctionGemma fine-tuning guide).
pip install torch transformers trl datasets accelerate
huggingface-cli login # accept the gemma license for google/functiongemma-270m-it
The Hub dataset stores messages/tools as JSON strings (Arrow cannot infer
the nested schema), and TRL's SFTTrainer needs a uniform struct schema, so
normalize first:
import json
from datasets import load_dataset
from transformers import AutoModelForCausalLM, AutoTokenizer
def normalize_messages(msgs):
out = []
for m in msgs:
n = {"role": m["role"], "content": m.get("content") or "", "name": None,
"tool_call_id": m.get("tool_call_id"), "tool_calls": None}
if m["role"] == "tool":
n["name"] = m["content"]["name"]
n["content"] = json.dumps(m["content"]["response"], ensure_ascii=False)
if m.get("tool_calls"):
n["tool_calls"] = [{"id": tc.get("id"), "type": tc.get("type", "function"),
"function": {"name": tc["function"]["name"],
"arguments": json.dumps(tc["function"]["arguments"], ensure_ascii=False)}}
for tc in m["tool_calls"]]
out.append(n)
return out
def rows_to_dataset(rows):
from datasets import Dataset
return Dataset.from_list([{
"messages": normalize_messages(r["messages"]),
"tools": json.dumps(r["tools"], ensure_ascii=False),
} for r in rows])
ds = load_dataset("Qrzysztof/ecommerce-chat-tool-calling", token=HF_TOKEN)["train"]
train_rows = [{"messages": json.loads(r["messages_json"]), "tools": json.loads(r["tools_json"])}
for r in ds if r["split"] == "train"]
train_ds = rows_to_dataset(train_rows)
import torch
from transformers import AutoModelForCausalLM
from trl import SFTConfig, SFTTrainer
model = AutoModelForCausalLM.from_pretrained("google/functiongemma-270m-it",
dtype=torch.bfloat16, attn_implementation="eager")
tokenizer = AutoTokenizer.from_pretrained("google/functiongemma-270m-it")
trainer = SFTTrainer(
model=model,
args=SFTConfig(
output_dir="functiongemma-ecommerce",
max_length=1024, # covers the longest sample + margin
packing=False, # keep tool calls intact (no cross-sample packing)
num_train_epochs=3,
per_device_train_batch_size=8,
learning_rate=5e-5,
lr_scheduler_type="constant",
warmup_steps=50,
bf16=True, # or fp16 on non-Ampere GPUs
eval_strategy="epoch",
report_to="none",
),
train_dataset=train_ds,
processing_class=tokenizer,
)
trainer.train()
TRL applies the FunctionGemma chat template with the per-sample tools
column; assistant_only_loss=True (default) masks everything but the model's
own turns, so it learns to emit tool calls — not to copy the schema.
ok = 0
for item in test_rows:
inputs = tokenizer.apply_chat_template(item["messages"][:-1], tools=item["tools"],
add_generation_prompt=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=256)
output = tokenizer.decode(out[0][len(inputs["input_ids"][0]):], skip_special_tokens=False)
expected = <expected tool name / args from expected_json>
ok += expected-tool-in-output and no-other-tool-in-output
trainer.push_to_hub("YOUR_USER/functiongemma-ecommerce")
Data
noise.py engine skips any token containing digits.template_id) and hold out
whole languages + (for the e-commerce set) whole schemas — that is the
only honest way to measure generalization.Training
packing=False for tool-calling data; packed sequences splice mid-call.max_length ≥ longest sample + a margin; ~1024 covers these datasets.Evaluation
Deployment
...-v2 (SafeTensors)...-tool-calling-v2