qwen2.5-0.5B-toolcall-mlx

qwen2.5-0.5B-toolcall-mlx

Full fused MLX weights for a tool-calling SFT study: Qwen2.5-0.5B-Instruct + LoRA (rank 16) fused, best-val checkpoint (iter-300, val 0.102). No adapter needed at inference. Apple Silicon via mlx-lm. Project + training code: see GitHub repo qwen-tool-sft-mlx (prepare_data.py, train.sh, eval.py, eval_benchmark.py, agent.py).

Tools

• get_weather: {"city": str (required), "unit": "celsius" | "fahrenheit" (optional)}
• calculator: {"expression": str} — emits canonical symbol form (144 / 12, keeps parens).

Call format: <tool_call>{"name": ..., "arguments": {...}}</tool_call>. Missing args -> ask (Which city should I look up?), never hallucinate. Same apply_chat_template(msgs, tools=...) everywhere.

Usage (MLX, Apple Silicon)

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("rautaditya/qwen2.5-0.5B-toolcall-mlx")
TOOLS = [
  {"type": "function", "function": {"name": "get_weather",
    "description": "Get current temperature and conditions for a city.",
    "parameters": {"type": "object",
      "properties": {"city": {"type": "string"},
                     "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
      "required": ["city"]}}},
  {"type": "function", "function": {"name": "calculator",
    "description": "Evaluate basic arithmetic.",
    "parameters": {"type": "object", "properties": {"expression": {"type": "string"}},
      "required": ["expression"]}}},
]
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Compute 47 * 13"}],
    tools=[TOOLS[1]], tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=100, sampler=make_sampler(temp=0.0)))
# <tool_call>
# {"name": "calculator", "arguments": {"expression": "47 * 13"}}
# </tool_call>

Training

• Base Qwen/Qwen2.5-0.5B-Instruct (full precision), LoRA rank=16, scale=20.0, dropout=0.05, lr=1e-4, batch 1, 16 layers, 800 iters. Trainable 1.187% (5.865M/494.033M). Data: 101 train / 11 valid / 4 test.
• Val: 2.717 (it1) -> 0.159 (50) -> 0.115 (100) -> 0.102 (300, BEST) -> 0.103 (400) -> 0.118 (800). Train loss 0.552 -> 0.027 kept falling after val plateaued = overfit past ~300.
• This upload = iter-300 fused (0000300_adapters.safetensors + base, fp16, ~942 MB).

Evals (measured on iter-800 adapter; fused iter-300 is strictly better val, spot-checked)

8-case strict eval.py: base score 0.688 / name 0.750 -> LoRA 1.000 / 1.000 (fixed dropped parens, 81/9 -> get_weather misfire, New York hallucination on clarify).

22-case held-out eval_benchmark.py (unseen cities/numbers, greedy):

Known limits

• Unseen city spellings can be mangled (Reykjavik -> Reykur, Lima -> Lisbon): train vocab was ~22 cities.
• Ambiguous weather phrasing can over-trigger clarification (Is it raining in Paris? -> asks city).
• Base always hallucinates "unit": "fahrenheit"; this model omits unit unless asked (correct).
• no_tool answers are short SFT strings, not grounded (trivia can be factually wrong); no RAG.
• MLX format only; for transformers/GGUF convert from base + adapter separately.