Qwen3-32B-ToolCall-GGUF

<p align="center"> <img src="assets/jatayu-labs-logo.png" width="170" alt="Jatayu Labs"/> </p> <h1 align="center">Qwen3-32B ToolCall — GGUF</h1> <p align="center"> <b>The Jatayu Labs agentic tool-calling model, packaged to run anywhere llama.cpp runs.<br/> One 48 GB GPU. One command. A full OpenAI-compatible tool-calling API.</b> </p> <p align="center"> <a href="https://huggingface.co/jatayulabs/Qwen3-32B-ToolCall">🧠 Source adapter + checkpoints</a> · <a href="#-serve-it-in-60-seconds">⚡ Serve in 60 seconds</a> · <a href="#-about-jatayu-labs">🪶 Jatayu Labs</a> </p>

---

🚀 What this model does

Give it a goal and a toolbox — it plans the sequence, calls the right tools with the right arguments, reads every result, recovers from failures, and reports back in the user's own language: English, हिंदी, or Hinglish.

<p align="center"> <img src="assets/orchestration-flow.png" alt="Verified orchestration flow" width="95%"/> </p>

Every transcript below was generated by this exact GGUF file, served with llama-server on 2× NVIDIA A40, temperature 0.1, unedited.

Episode 1 — end-to-end order (English)

USER: Order 2 packets of Amul milk and 1 brown bread on Zepto. Pay cash on delivery.

[1] → search_multiple_products({"queries": ["Amul milk", "brown bread"]})
[2] → update_cart({"items": [{"product_id": "ZP-MILK-001", "quantity": 2},
                             {"product_id": "ZP-BREAD-007", "quantity": 1}]})
[3] → get_payment_methods({})
[4] → create_order({"payment_method": "COD"})

MODEL: Your order has been confirmed! 🛒
       Order ID: ZO-88121
       Items: 2 × Amul Taaza Toned Milk 500ml (₹29 each)
              1 × Britannia Brown Bread 400g (₹50)
       Total: ₹108 | Payment: Cash on Delivery
       Estimated delivery: 9 minutes.

One request → four correct tool calls → a confirmation where every number (order ID, prices, ETA) comes from tool results. Nothing invented.

Episode 2 — same task, in Hinglish

USER: yaar zepto se 2 amul doodh aur ek brown bread mangwa de, cash on delivery rakhna

[1] → search_multiple_products({"queries": ["Amul doodh", "brown bread"]})
[2] → update_cart({...})  [3] → get_payment_methods({})  [4] → create_order({"payment_method": "COD"})

MODEL: Your order has been confirmed! 🛒  Order ID: ZO-88121 · Total ₹108 ·
       COD · delivery in ~9 minutes.

Casual romanized Hindi is a first-class input — the model even carried the user's own word ("Amul doodh") into the search query.

Episode 3 — Hindi in, Hindi out

USER: कृपया Zepto से 2 अमूल दूध और 1 ब्राउन ब्रेड ऑर्डर करें। भुगतान कैश ऑन डिलीवरी होगा।

[1-4] → identical four-step tool plan

MODEL: आपका ऑर्डर स्वीकृत हो गया है!
       ऑर्डर ID: ZO-88121
       कुल राशि: ₹108 · भुगतान विधि: कैश ऑन डिलीवरी (COD)
       अनुमानित डिलीवरी समय: 9 मिनट
       हम जल्द से जल्द आपके पास डिलीवर करेंगे! 🚚

Devanagari in → Devanagari out. No language flag, no extra prompting.

Episode 4 — error recovery

USER: Order 2 Amul milk on Zepto, cash on delivery.

[3] → create_order({"payment_method": "cod"})
      ← {"status": "error", "reason": "COD not available for this store. Available: UPI, Zepto Cash"}

MODEL: The Cash on Delivery (COD) option is not available for this store. You can
       proceed using UPI or Zepto Cash. Would you like to update your payment preference?

A payment failure is surfaced and handed back to the user — no silent retries, no fake success.

Measured results (this file, this hardware)

⚡ Serve it in 60 seconds

hf download jatayulabs/Qwen3-32B-ToolCall-GGUF qwen3-32b-toolcall-Q4_K_M.gguf --local-dir models

llama-server -m models/qwen3-32b-toolcall-Q4_K_M.gguf \
    --host 0.0.0.0 --port 8000 --jinja -ngl 99 -c 8192

--jinja activates the embedded tool-calling chat template. You now have a standard OpenAI-compatible endpoint:

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "What is the weather in Delhi right now?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "description": "Get current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}},
                   "required": ["city"]}}}]
}'
# → finish_reason: "tool_calls", get_weather({"city": "Delhi"})

Works with any OpenAI SDK — point base_url at your server and pass tools as usual.

File

---

🔬 How we trained it

Trained on NVIDIA A100 GPUs with LLaMA-Factory, using LoRA on top of Qwen3-32B. This GGUF is the step-600 release checkpoint (best train loss, 0.070) merged into the base weights.

<p align="center"> <img src="assets/training-loss.png" alt="Training loss curve" width="90%"/> </p>

Epoch 1 learns the format of structured tool calls (plateau ~0.45), a sharp drop at the epoch boundary marks format mastery (0.44 → 0.23), epoch 2 refines planning, and the model locks in at 0.070 by step 600. Step 800 showed early overfitting (0.077), so step 600 is the release.

---

🪶 About Jatayu Labs

<img src="assets/jatayu-labs-logo.png" width="90" align="right" alt="Jatayu Labs"/>

Jatayu Labs is an Indian AI lab building agentic intelligence for a billion users — in their own languages.

• 🔧 Fine-tuned LLMs for agentic tool calling and orchestration
• 🎯 Specialised small language models for focused production tasks
• 🎙️ Voice-modality models — speech in, speech out
• 🇮🇳 Indic languages first — Hindi, Hinglish, and regional languages as first-class citizens

Jatayu — the bird of Indian epic who flew fastest and acted first.

License: Apache 2.0 · Contact: via the Community tab