tooltuned-qwen-3.5-4b

tooltuned-qwen-3.5-4b

TL;DR

LoRA (rank 16) fine-tune of Qwen/Qwen3.5-4B for tool-calling, trained on Salesforce/xlam-function-calling-60k with Unsloth + TRL.

• Result on BFCL V4: 79.0% (base 87.3%, -8.3pp)
• Adapter: sukhrobnurali/tooltuned-qwen-3.5-4b
• Format: bf16 LoRA adapter (Unsloth advises against 4-bit quant for Qwen 3.5)

BFCL V4 results

Gate disclosure (v1.0): This adapter is published below the +3pp BFCL gate defined in the project brief (delta -8.30pp on the in-tree V3 evaluator). The regression is concentrated in irrelevance / live_irrelevance categories -- see ADR 0006 for the locked diagnosis and the Phase 3.5 remediation spec (deferred for v1.0).

Per-category breakdown

n=458, evaluated 2026-05-13

Training data

• Sources: xlam (Salesforce/xlam-function-calling-60k)
• Samples used: 10,000
• Validation fraction: 0.05
• Held-out fraction: 0.05
• Thinking-mode strategy: preserve (preserves Qwen 3.5's default reasoning trace; xLAM rows have no <think> content so the three strategies converge in practice)

Training procedure

Supervised fine-tuning via Unsloth's FastLanguageModel + TRL's SFTTrainer. Adapter only — base weights are frozen. Single A100 (40 GB), bf16, gradient checkpointing on.

Hyperparameters

Intended use

Function calling / tool use in chat agents. The adapter pairs with the base Qwen 3.5 4B chat template; pass tool schemas in the system prompt and the model emits <tool_call> blocks (or XML-tagged <function=...> calls; the inference helper parses both).

Out of scope

• Non-English instruction following (xLAM is English-only).
• Long-context tool dialogues beyond 2,048 tokens — the adapter was trained at that sequence length.
• Safety-critical decisions. The adapter inherits Qwen 3.5's safety profile, no additional alignment was applied.

Limitations

• LoRA rank 16 is a known-safe default, not an ablated optimum. Higher ranks may move the BFCL number further; rank ablations are a stretch goal.
• BFCL holds out one slice of tool-calling behavior; performance on task families outside that distribution (multi-turn agentic loops, fully novel APIs) is not directly measured.

License

Apache-2.0, matching the base model Qwen/Qwen3.5-4B.

Reproduction

Source: https://github.com/sukhrobnurali/tooltuned-qwen. Pinned versions live in pyproject.toml; the lockfile (uv.lock) is the reproducibility contract.

git clone https://github.com/sukhrobnurali/tooltuned-qwen
cd tooltuned-qwen
uv sync
# Run on Colab Pro A100; see notebooks/colab_main.ipynb

Training curves: https://wandb.ai/sukhrob-production/tooltuned-qwen.

Citation

@misc{nurali_tooltuned_qwen_2026,
  author       = {Sukhrob Nurali},
  title        = {tooltuned-qwen-3.5-4b: a tool-calling LoRA for Qwen 3.5 4B},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sukhrobnurali/tooltuned-qwen-3.5-4b}}
}

Author

• Sukhrob Nurali — [email protected]
• Hugging Face: sukhrobnurali
• GitHub: sukhrobnurali