qwen2.5-3b-function-calling-lora

Model Card for function-calling-lora

LoRA adapter fine-tuned on top of Qwen/Qwen2.5-3B-Instruct for structured function/tool calling. Trained with TRL SFTTrainer and PEFT LoRA (r=16, lora_alpha=32, targeting q_proj/v_proj).

This is an adapter, not a merged model — you need the base model plus peft to load it.

Supported tools

• get_weather(location, unit)
• get_news(topic, limit)
• calculate(expression)

Prompt format

The model expects a custom pseudo-chat format (not the standard HF chat template):

<|system|>
{system prompt}

<|tools|>
{tools.json as a JSON array}

<|user|>
{user message}

<|assistant|>

The model completes the <|assistant|> block with either:

• a JSON tool call: {"name": "...", "arguments": {...}} (it was also trained on code-fenced, <tool_call>...</tool_call> XML-style, and OpenAI-style {"tool_calls": [...]} variants), or
• a plain natural-language reply when no tool applies or required arguments are missing.

Multi-turn conversations extend the same format with additional <|user|> / <|assistant|> / <|tool|> blocks.

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-3B-Instruct"
ADAPTER = "navenduk75/qwen2.5-3b-function-calling-lora"

base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.float32)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
model = PeftModel.from_pretrained(base_model, ADAPTER).merge_and_unload()

prompt = """<|system|>
You are a function-calling model. If the user's request matches one of the available tools, respond ONLY with a JSON object describing the tool to call and its arguments. If no tool applies, or required information is missing, respond in plain natural language instead.

<|tools|>
[{"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["location"]}}]

<|user|>
What's the weather in Berlin in celsius?

<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt")
output_ids = model.generate(**inputs, max_new_tokens=200, do_sample=False, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training procedure

Trained on a synthetic dataset covering: clean single-turn calls, noisy/typo'd phrasing, multiple tool-call output formats, multi-turn conversations with synthetic tool results, and off-topic/ambiguous prompts that should not trigger a tool call. This model was trained with SFT.

Framework versions

• PEFT 0.20.0
• TRL: 1.10.0
• Transformers: 5.15.0
• Pytorch: 2.13.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Citations

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}