smol-tools-4b

smol-tools-4b — Agentic Tool-Use Model

A 4B parameter text-only model fine-tuned for reliable tool selection, structured JSON output, and knowing when NOT to use tools. Built on Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled, trained with LoRA on 6,855 quality-filtered synthetic examples.

Architecture: Qwen3_5ForCausalLM (text-only, no vision encoder). Vision weights from the base model have been stripped — this model is purpose-built for text-based tool calling.

Need longer context? See smol-tools-4b-16k (16K context) and smol-tools-4b-32k (32K context) for multi-turn agent workflows.

Available Formats

GGUF files available in enfuse/smol-tools-4b-GGUF.

Results (200-example held-out eval)

Per-Scenario Breakdown

Capabilities

• Tool selection: Picks the right tool(s) from a provided set with 95.5% F1
• Structured output: Produces valid <tool_call>{"name": "...", "arguments": {...}}</tool_call> JSON — 100% validity
• Tool refusal: Correctly answers directly when no tool is needed — 100% accuracy
• Multi-tool: Handles parallel and sequential multi-tool scenarios perfectly
• Reasoning: Generates chain-of-thought reasoning in <think> tags before acting

Available Tools (training set)

The model was trained with these 15 tools but generalizes to new tool schemas provided at inference:

web_search, get_webpage, execute_python, read_file, write_file, list_directory, send_email, get_current_datetime, calculate, translate, get_weather, create_calendar_event, database_query, http_request, shell_command

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "enfuse/smol-tools-4b",  # or local path
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("enfuse/smol-tools-4b", trust_remote_code=True)

tools = [
    {"type": "function", "function": {
        "name": "web_search",
        "description": "Search the web for information",
        "parameters": {"type": "object", "properties": {
            "query": {"type": "string"}
        }, "required": ["query"]}
    }}
]

messages = [
    {"role": "system", "content": "You are a helpful assistant with access to tools."},
    {"role": "user", "content": "What's the latest news about SpaceX?"},
]

prompt = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))

With vLLM (faster)

from vllm import LLM, SamplingParams

llm = LLM(model="enfuse/smol-tools-4b", dtype="bfloat16", max_model_len=4096, enforce_eager=True)
sampling = SamplingParams(max_tokens=2048, temperature=0.1, stop=["<|im_end|>"])
outputs = llm.generate([prompt], sampling)

Output Format

The model responds with optional thinking followed by tool calls or a direct answer:

With tool call:

<think>
The user wants to search for SpaceX news. I should use the web_search tool.
</think>

I'll search for the latest SpaceX news for you.

<tool_call>
{"name": "web_search", "arguments": {"query": "latest SpaceX news"}}
</tool_call>

Without tool call (direct answer):

<think>
This is a general knowledge question I can answer directly without any tools.
</think>

The capital of France is Paris. It has been the capital since...

Training Details

Data Pipeline

1. Teacher model: Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled generated synthetic tool-use conversations
2. Quality filtering: Removed examples with malformed JSON, missing tool calls, or incorrect tool usage (5,000 → 4,578)
3. Targeted generation: Generated 2,277 additional examples focusing on reasoning_heavy and complex_multi_step scenarios with explicit <think> tag prompting
4. Combined dataset: 6,855 examples across 7 scenario types

What Worked (Experiment Log)

smol-tools Family

All models share the same base architecture, tool schema, and output format. Choose based on your context length needs:

How to choose:

• 4K (this model): Single-turn tool calls, short tool outputs — highest accuracy, lowest memory
• 16K: Multi-turn conversations (5-10 rounds), moderate tool outputs — also available in GGUF quantized formats
• 32K: Extended agent sessions (10-20 rounds), large tool outputs — also available in GGUF quantized formats

When to Use This Model

• You're building an agent or copilot on the edge — local devices, Jetson, phones, on-prem servers with limited GPU
• You need thousands of tool-calling inferences per minute cheaply — a 4B model serves 10–50x faster than a 70B at a fraction of the cost
• You need structured output you can trust — 100% JSON validity means no crashed pipelines from malformed tool calls
• You're tired of paying per-token API costs for tool-use that a small local model can handle

When NOT to Use This Model

• If your agent needs multi-turn conversations or long tool outputs, use smol-tools-4b-16k or smol-tools-4b-32k instead
• If you need GPT-4-level complex multi-step planning (our weakest category at F1=0.818), use a bigger model
• If latency and cost don't matter, just call a frontier API — they'll outperform any 4B model on hard reasoning
• If your use case requires tools not seen during training, test carefully — the model generalizes to new tool schemas but hasn't been validated on every possible tool type

Limitations

• complex_multi_step scenarios (F1=0.818) remain the weakest — the model sometimes struggles with multi-step planning involving 3+ chained tools
• No thinking rate in evaluation (0%) — the model reasons but doesn't always use explicit <think> tags at low temperature
• Trained on synthetic data only — real-world tool-use patterns may differ
• Inherits Qwen3.5-4B base model limitations (context window, knowledge cutoff)

Hardware

• Training: 1× NVIDIA H200 NVL (141 GB HBM3e)
• Inference (BF16): Any GPU with ≥10 GB VRAM
• Inference (Q8_0 GGUF): Any device with ≥6 GB RAM — Jetson Orin NX, consumer GPUs
• Inference (Q4_K_M GGUF): Any device with ≥4 GB RAM — Jetson Orin Nano, phones, Raspberry Pi 5

Attribution

• Base model: Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled by Jackrong
• Training framework: TRL + PEFT by HuggingFace
• Inference: vLLM