qwen3-14b-search-plan-act

Qwen3-14B, GRPO-trained on search_plan_act

Full-weight checkpoint (LoRA merged) of Qwen/Qwen3-14B, trained with GRPO (via TRL's multi-turn environment_factory tool-calling API) on search_plan_act: a procedurally-generated, closed-book multi-turn task requiring the model to search for facts, read_record/update_record/ link_records on retrieved entities, and finish, looping through tool calls until the goal is achieved or it gives up. Reward decomposes into outcome / grounding / stop-behavior components, with an explicit adversarial-pilot check that no reward-hacking strategy (reckless guessing, spamming searches, never finishing) scores above genuine goal-directed play.

Training: LoRA (r=32, alpha=64) on all attention+MLP projections, GRPO via TRL 1.9.2 + vLLM colocate generation, 250 steps, effective batch size 32, 4x GH200 GPUs. Memory-retuned specifically for this model size (halved per-device batch, doubled gradient accumulation to preserve the same effective batch, and a modest vLLM KV-cache allocation bump) -- a 14B model needs roughly two full weight copies coexisting on one GPU under colocate mode (one for training, one for vLLM's inference engine), which the smaller-model default settings didn't budget for.

Evaluation

Evaluated pre-RL (base Qwen/Qwen3-14B) vs post-RL (this checkpoint) on held-out real-world and synthetic benchmarks, all using vLLM-accelerated generation with the model's own chat template (apples-to-apples generation path in both rows):

Honest finding: this 14B checkpoint shows only a small gain on search_plan_ood and is flat everywhere else, including BALROG (where smaller Qwen3 sizes showed a small real gain). Compare against the sibling bmonikraj/qwen3-8b-search-plan-act checkpoint, which showed much stronger, more consistent transfer under the identical training recipe -- RL transfer here is genuinely model-specific, not simply a function of scale.

Files

• model.safetensors, config.json, tokenizer.json, tokenizer_config.json, generation_config.json, chat_template.jinja -- the full merged checkpoint.
• adapter/ -- the raw LoRA adapter (pre-merge), kept for reference.