florin-tau2-retail-30b

Florin-tau2-Retail-30B (LoRA adapter for Qwen3-30B-A3B-Thinking-2507)

DRAFT — private staging copy. Do not make public without explicit approval.

An attention-only LoRA adapter (r=64, α=128, dropout 0.05, q/k/v/o across all 48 layers; 53.5M trainable parameters) that fine-tunes Qwen/Qwen3-30B-A3B-Thinking-2507 for the τ²-bench retail customer-service domain.

Results (τ²-bench retail, 114 tasks × 4 trials, seed 300, commit a2c0247 / v1.0.1 task set)

Computed with tau2-bench's own compute_metrics; trajectories in the companion dataset repo.

Ties RAFT-30B-A3B (the merged 30B fine-tune precedent) on pass^1; trails it on pass^2–4. User simulator: gpt-5.2 (reasoning_effort=low), per the leaderboard's documented protocol.

Provenance

• Base model: Qwen/Qwen3-30B-A3B-Thinking-2507 (Apache-2.0). All credit for the base capabilities to the Qwen team.
• Training data: 2,117 synthetic retail conversations (SFT batch-2 pool) generated with open-weight models only — teacher agent Qwen3.5-397B-A17B, user simulator Qwen3-235B-A22B-2507 — via the tau2-bench orchestrator on a fully synthetic retail database whose entities (users, products, orders) are disjoint from the τ²-bench evaluation database. Trajectories were accepted only if they passed tau2's deterministic DB-state check. No proprietary-model outputs are in the training data.
• Training: LLaMA-Factory 0.9.5, 2 epochs, lr 1e-4 cosine (Adam β 0.9/0.95, warmup 0.03), global batch 8, cutoff 16,384 tokens, bf16, Liger fused cross-entropy, single H100 80GB, 530 optimizer steps, ~3.0 h. Loss on assistant turns only; teacher reasoning kept as <think> blocks on trained turns.
• Checkpoint selection: epoch-2 checkpoint chosen on a held-out 248-task synthetic dev split (2 trials, deterministic DB-check grading): dev pass^1 98.0 vs 96.98 for epoch 1. The real 114-task set was never used for any tuning or selection decision.

Serving (reproduces the official-run configuration)

vllm serve Qwen/Qwen3-30B-A3B-Thinking-2507 \
  --served-model-name qwen3-30b-a3b-thinking-2507 \
  --enable-lora --max-lora-rank 64 \
  --lora-modules ep2=<this adapter> \
  --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3 \
  --max-model-len 32768 --max-num-seqs 16 --gpu-memory-utilization 0.90
# agent sampling used in the official run: temperature 1.0, top_p 0.95, max_tokens 8192