restraint-7b

Restraint-7B

A 7B tool-use model post-trained to answer knowledge questions without calling tools — and to call them precisely when they're needed.

SFT on 450 expert ReAct traces, then a corrective SFT pass that fixed a data bug which had collapsed the model's restraint behavior (all 150 original restraint demos shared one canned <think> line and omitted the system prompt, so the model never learned a real decision boundary).

The problem it fixes

Tool-capable models over-tool. After format SFT, the model calls tools on questions it already knows — wasted latency, wasted tokens, unnecessary failure surface. Restraint-7B is trained to hold back: "What is 6 squared?" gets a direct answer, not a calculator call.

Results

50-task stratified eval (same task IDs for both models, temperature 0, real deterministic tool executor, canonical is_correct scoring — no LLM judge):

Paired result: restraint 0% → 94% while accuracy rose 78% → 92% — fewer unnecessary calls means fewer failure surfaces, not a capability trade-off.

Failure audit (v2): of the 4 misses, 3 are scorer strictness on definitional restraint answers (semantically correct, wording differs from the reference string); 1 is a genuine held-out error-recovery failure.

Eval harness, task suite, ground-truth repair diff, and raw traces: github.com/harneet2512/Codetune. Traces are regenerated per release; scoring is deterministic.

Usage

Raw-completion format — the model was trained on this prompt shape, not chat markup:

from llama_cpp import Llama  # or transformers / vLLM on the merged weights

prompt = """You are a helpful assistant with access to the following tools:

[{ "name": "calculator", ... }]

To use a tool, write a <tool_call> block with JSON. You will receive the result in an <observation> block.

Think step by step inside <think></think> tags. When you have enough information, give your final answer inside <answer></answer> tags.
If a tool returns an error, try a different approach.

User: What is 37 multiplied by 576?
"""

Output contract: <think>…</think> → optional <tool_call>{"name":…,"arguments":{…}}</tool_call> → <observation> is supplied by your harness → final <answer>…</answer>.

Important: when serving via llama.cpp /completion, add </tool_call> (and <observation>) to the stop list — otherwise the model continues past its own tool call and fabricates an observation instead of waiting for the real one. (We learned this the hard way; it silently corrupted an entire eval run.)

GGUF

restraint-7b-v2-q4km.gguf (4.5GB) runs in llama.cpp / Ollama / LM Studio. Verified on a 6GB RTX 2060 at ~39 tok/s.

Training details

• Base: Qwen/Qwen2.5-7B-Instruct
• Stage 1: QLoRA SFT, r=16 α=32, 450 traces (200 single-tool / 150 restraint / 100 multi-step)
• Stage 2 (v2): corrective QLoRA SFT on the merged stage-1 weights — 350 examples (150 restraint with varied reasoning text + full tool-definition system prompt, 200 tool/multi-step), 2 epochs, lr 1e-4, ~13 min on L4
• Post-train sanity check: asserts LoRA B-matrix magnitude is non-trivial (catches the silent no-op-adapter failure our first GRPO run produced)
• Training + eval code: train/, tooltune/, scripts/ in the repo

Limitations

• The 5 tools are a deterministic simulated set (calculator, wikipedia, weather, code_executor, unit_converter) — the suite measures tool selection, format, and faithfulness to tool output, not real API integration. Generalization to live API ecosystems is unverified.
• 50-task eval sample: wide confidence intervals (±~14pp at 95% CI). Tier 4 held-out n=2 is anecdotal.
• Tier 1–3 tasks are in-distribution relative to training; only tier 4 is a true held-out split.