Downloads · 30 days
9
13% of all-time downloads
abdelstark/llama-3.1-nemotron-nano-8b-xlam-tool-calling-lora
llama-3.1-nemotron-nano-8b-xlam-tool-calling-lora is a text generation model from abdelstark. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as other.
Downloads · 30 days
9
13% of all-time downloads
All-time downloads
71
Public
Repo size
185 MB
Likes
0
Public
Click a slice to open those files.
.safetensors168 MB · 90%
From the Hugging Face model README
Built with Llama.
A QLoRA adapter for nvidia/Llama-3.1-Nemotron-Nano-8B-v1 that turns free-form user requests plus a set of JSON tool schemas into exactly one schema-valid JSON tool call — no prose, no markdown fences, no explanations.
Trained and evaluated end to end with
sommelier, a reference pipeline
for reproducible tool-calling fine-tuning (deterministic data preparation,
prompt digests, completion-only loss, deterministic evaluation, and a
digest-gated base-vs-adapter comparison). This repository contains the
adapter weights, tokenizer metadata, and the machine-readable evaluation
evidence for the exact run that produced them (nemotron-8b-full-3).
Base model vs. this adapter on the 1,000-example held-out test split of
abdelstark/sommelier-xlam-single-call-splits.
Both evaluations used byte-identical prompts (prompt-set digest
a0da8fa2…), greedy decoding (temperature 0.0, max_new_tokens 512), and
the same conservative parser (sommelier.parser.v1) that counts every
parse failure as a metric failure. The comparison is only written when
config, test-split, prompt-set, parser, and decoding digests all match.
| Metric | Base | Adapter | Delta |
|---|---|---|---|
| valid_json_rate | 0.9160 (916/1000) | 1.0000 (1000/1000) | +0.0840 |
| function_name_accuracy | 0.9110 (911/1000) | 0.9960 (996/1000) | +0.0850 |
| argument_exact_match | 0.7070 (707/1000) | 0.8760 (876/1000) | +0.1690 |
| argument_f1 | 0.7569 | 0.9291 | +0.1722 |
| full_call_exact_match | 0.7050 (705/1000) | 0.8740 (874/1000) | +0.1690 |
Metric definitions: valid_json_rate — output parses into exactly one
{"name": …, "arguments": …} call; argument_f1 — micro-F1 over
arguments flattened to dotted key paths with canonical scalar JSON values
(lists compared by index); exact-match metrics use canonical-JSON equality.
Numerators and denominators for every metric, plus raw per-example
records, are in reports/.
The adapter was trained and evaluated with one fixed prompt policy. Reproduce it exactly for best results:
system: the instruction below, then two newlines, then
Available tools: and the tool schemas serialized as canonical JSON
(sorted keys, compact ,/: separators):
You are a tool-calling model. Select the correct tool and return only
the JSON tool call. Do not include explanations.
Available tools:
[{"description":"…","name":"…","parameters":{…}}]
user: the raw request text.
The model answers with a one-element canonical JSON array:
[{"arguments":{…},"name":"…"}].
import json
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE = "nvidia/Llama-3.1-Nemotron-Nano-8B-v1"
ADAPTER = "abdelstark/llama-3.1-nemotron-nano-8b-xlam-tool-calling-lora"
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()
SYSTEM = (
"You are a tool-calling model. Select the correct tool and return only "
"the JSON tool call. Do not include explanations."
)
tools = [
{
"name": "lookup_weather",
"description": "Look up the current weather for a city.",
# xlam-style flat parameter map — the schema shape this adapter was
# trained on. JSON-Schema-style {"type": "object", "properties": …}
# tools are out of distribution and typically fail to parse.
"parameters": {"city": {"description": "Name of the city.", "type": "str"}},
}
]
tools_json = json.dumps(tools, separators=(",", ":"), sort_keys=True)
messages = [
{"role": "system", "content": f"{SYSTEM}\n\nAvailable tools:\n{tools_json}"},
{"role": "user", "content": "What is the weather in Paris today?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.no_grad():
output = model.generate(**inputs, do_sample=False, max_new_tokens=512)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# [{"arguments":{"city":"Paris"},"name":"lookup_weather"}]
Note add_special_tokens=False: the rendered chat template already
contains the BOS token.
QLoRA with completion-only loss: every prompt token is masked with the ignore index and loss is computed only on the assistant tool-call tokens, with the prompt/target token boundary proven per example (training fails rather than falling back to full-sequence loss).
| Setting | Value |
|---|---|
| Base model | nvidia/Llama-3.1-Nemotron-Nano-8B-v1 (revision main) |
| Quantization (training) | NF4 4-bit, double quantization, bfloat16 compute |
| LoRA | r=16, alpha=32, dropout=0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Epochs | 2 (1,876 optimizer steps) |
| Batch | 4 per device × 4 gradient accumulation (effective 16) |
| Learning rate | 2e-4, cosine schedule, warmup ratio 0.03 |
| Max sequence length | 4,096 (longest rendered example: 2,166 tokens) |
| Gradient checkpointing | enabled (non-reentrant) |
| Seed | 42 |
| Tokens seen | 16,668,132 |
| Train loss | 1.085 → 0.042 |
| Eval loss | 0.0381 (epoch 1) → 0.0303 (epoch 2) |
| Hardware | 1× NVIDIA L40S (peak 26,306 MiB) |
| Training wall time | 10,996 s (~3.05 h) |
| Stack | torch 2.12.1, transformers 5.12.1, peft 0.19.1, bitsandbytes 0.49.2 |
Full hyperparameters and the resolved run configuration are in
reports/config.resolved.yaml; per-step
metrics in reports/training_metrics.jsonl.
Trained on the train split (15,000 examples) of
abdelstark/sommelier-xlam-single-call-splits,
a filtered, deduplicated, deterministically split derivative of
Salesforce/xlam-function-calling-60k
(CC-BY-4.0) containing only single-call examples. The validation split
(1,000) was used for eval loss only; the test split (1,000) was never used
for gradient updates. A normalized query appears in exactly one split.
nemotron-8b-full-33a8fb0d8b4f5b4c6bcde370a51b0111598814e28df78ba8e6c2c25a907e5958bdb8dd82f…; prompt set digest: a0da8fa2…reports/comparison_report.json,
per-model evaluation reports, runtime metadata (stage timings, peak GPU
memory, cost marked unavailable rather than implied zero)Intended for research and engineering evaluation of schema-valid single-tool-call generation, and as a reference result for the sommelier pipeline.
"parameters": {"<param>": {"description": …, "type": …}});
JSON-Schema-style {"type": "object", "properties": …} schemas are out
of distribution and typically produce unparseable calls.Built with Llama.
If you use this adapter, please cite the source dataset:
@article{liu2024apigen,
title={APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets},
author={Liu, Zuxin and Hoang, Thai and Zhang, Jianguo and Zhu, Ming and Lan, Tian and Kokane, Shirley and Tan, Juntao and Yao, Weiran and Liu, Zhiwei and Feng, Yihao and others},
journal={arXiv preprint arXiv:2406.18518},
year={2024}
}