helmsman-qwen3-1.7b-v1

Helmsman — Qwen3-1.7B adaptive-effort tool-caller (v1)

The System-2 escalation tool-caller for the ZeroClaw browser agent, LoRA-fine-tuned from mlx-community/Qwen3-1.7B-bf16 on CALIBER. It reads the "log pose" — deciding how much to think — and steers effort: deliberate on ambiguous / multi-step turns, act directly on unambiguous single-tool turns, and abstain / ask / refuse-destructive correctly.

Intended use & deployment posture

• This is an escalation brain, NOT a low-latency router. The <150 ms per-message hot path in ZeroClaw stays a heuristic + MiniLM classifier (the 0.5B ONNX decoder is 9.7 s WASM-q8 / 14.9 s WebGPU per message; a thinking model is strictly slower). Helmsman runs where deliberation is worth seconds: local Ollama / llama.cpp, opt-in, or as a shadow-gold oracle.
• Adaptive think-only-when-hard is itself a latency control: System-1 turns emit no <think> and short output, so the common case stays fast even on the escalation model.
• Output format is deployed Hermes-XML (<think>…</think> + <tool_call>{…}</tool_call>, tool results as role:tool) — no train/serve skew.

Training

• Stage 1 — trace-SFT (LoRA): rank 16 / alpha 32 / 16 layers on q,k,v,o; max_seq_length 4096 (multi-turn + thinking rows truncate at the shared config's 1024); config-caliber-1.7b.yaml.
• Stage 2 — preference refinement: reject-sampling (ReST) → DPO on {chosen = correct adaptive-effort trajectory, rejected ∈ {over-think-on-System1, under-think-on-System2, wrong-tool, hallucinated-args, failure-to-abstain, broken-dependency}}, with an auxiliary NLL term to preserve Hermes formatting. GRPO is a future stage.
• Distill-then-on-policy: a 1.7B must be distilled from a stronger teacher before on-policy refinement helps at this scale (SOD 2605.07725, D3-Gym 2604.27977).

Evaluation

Measured on the matched 155-case multi-turn benchmark (scripts/finetune/eval/trajectory-scorer.js), Helmsman-v1 (LoRA) vs the Qwen3-1.7B base:

¹ fraction of act-directly turns carrying a non-empty <think> block. Qwen3 emits an empty <think></think> in non-thinking mode — that is zero reasoning (System-1) and is not counted.

Passed the confirm-gate: Δaccuracy +18.1pp (≥+5pp), abstention preserved, overThink ≤0.25, falseEscalation ≤0.15, malformed ≤0.10. The base does real substantive over-thinking (0.70); Helmsman deliberates on only ~20% of easy turns and otherwise acts directly — adaptive effort 0.43 → 0.90.

Honest limitations: multi_step dependency chains are still weak (0.04) — exact multi-turn tool-call-set match across ≥2 dependent steps is the hard frontier for a 1.7B on a 383-row v1 corpus. External BFCL-v4 / tau2-bench numbers are not yet run. This is a v1 that wins decisively on adaptive-effort + abstention + format discipline, not yet on long multi-step orchestration.

The production in-browser router pointer (promoted-pointer.json) stays verdict: INITIAL and is untouched by this release — Helmsman ships as the System-2 escalation brain (Ollama/local/ shadow), not the <150 ms hot path.

Limitations

Small-model reasoning ceiling; synthetic-distribution risk (mitigated by external gating + an anti-collapse real-data floor); the WHEN2TOOL scope note (value is effort suppression + plan / grounding / abstain, not "reasoning decides the call"). See the dataset card §8.