qwen3-8b-code-pkpo

Qwen3-8B Code PKPO

This repo contains a Qwen/Qwen3-8B-Base derivative trained for a small agentic coding experiment using the shared tool path in agent_core.py and shipped_tool.py.

Method

• Base: Qwen/Qwen3-8B-Base.
• Prompt/template: custom <think>...</think><answer>...</answer> template saved in the tokenizer. The generation prompt ends with Assistant: <think>.
• Tool protocol: no system role; instructions are merged into the first user message; strict user/assistant alternation; plain-text Tool type and Tool query calls.
• Training data: deepmind/code_contests train split only, filtered to old stdin/stdout problems. The LiveCodeBench eval subset is not used for training.
• Reward: binary hidden-test pass/fail.
• PKPO: sloo_minus_one from the paper for k >= 2; centered k=1 rewards for the first and final stages. No GRPO-style reward normalization is applied.
• Schedule actually run: [1, 8, 1] (shipped: stage1_group3).

The run was intentionally small to fit the free-credit budget and deadline. Results should be treated as a reproducible experiment, not a leaderboard model.

Results

Evaluation uses livecodebench/code_generation_lite v6, a fixed subset saved at eval/eval_subset.json, temperature 1.0, and the same one-turn tool path used for training.

Raw files:

• eval/baseline_results.json
• eval/final_results.json
• eval/eval_subset.json

Usage

Serve with vLLM:

vllm serve bk1dr/qwen3-8b-code-pkpo --trust-remote-code --max-model-len 8192

Run the shipped tool:

python shipped_tool.py --base-url http://127.0.0.1:8000/v1 --model bk1dr/qwen3-8b-code-pkpo --max-turns 1 --cp < problem.txt

Run Notes

Run pkpo_20260709T184830Z: full PKPO schedule k=1->8->1 with a LoRA checkpoint after every group. The shipped weights are checkpoint 'stage1_group3', selected by validation on the fixed eval subset (per-checkpoint pass@1: {"after_sft": 0.027777777777777776, "stage1_group3": 0.05555555555555555, "stage2_group1": 0.027777777777777776}). The full-schedule endpoint regressed on the subset (see eval/full_schedule_endpoint_results.json); intermediate checkpoint selection is part of the documented training procedure. Selection+merge elapsed 13.3 min on one H100.