OPSD-Qwen3.5-9B-LoRA-Medium-iter31

Qwen3.5-9B OPSD-PI Medium LoRA — iteration 31

This repository contains the adapter-only final snapshot of the sequential 1,024-instance Medium-PI OPSD run. The frozen Qwen3.5-9B base model is not included.

Training lineage

The adapter is cumulative across 32 updates numbered 0 through 31, with a global batch size of 32:

1. updates [0, 16) started from the pinned Qwen3.5-9B Base plus an empty LoRA and consumed the first 512-instance shard;
2. updates [16, 32) resumed the iteration-15 Medium adapter and consumed a disjoint latter 512-instance shard.

The two dataset manifests each contain 512 unique instance IDs, have zero ID overlap, and have a 1,024-ID union. This is therefore the final adapter of the sequential 512 + 512 run. It is not a fresh run over a newly mixed or reshuffled 1,024-row file, so it should not be presented as weight-equivalent to that different training order.

The last batch was replayed from its preserved rollout after an interrupted attempt. The replay was verified to contain the same 32 instances, group IDs 480 through 511; all 32 groups completed and all 92 retained trajectory branches had non-empty trainable masks.

Adapter format

adapter/checkpoint_iter_0000031.safetensors is a raw Slime/Megatron adapter, not a PEFT directory. It contains 168 BF16 tensors ending in .lora_A or .lora_B (84 complete A/B pairs):

• base model: Qwen/Qwen3.5-9B
• base revision: c202236235762e1c871ad0ccb60c8ee5ba337b9a
• LoRA rank / alpha / dropout: 64 / 128 / 0
• scaling: 2.0
• LoRA parameters: 108,134,400
• SHA-256: 382b7a2f0b23cfb97d949aeedfabc7d1c21f556e9ae2e043a09eb50d920650de

PeftModel.from_pretrained is not compatible with this raw tensor naming and fusion layout. Load it with the Qwen3.5 model plugin in the OPSD training code. The file excludes the base weights, optimizer state, full Megatron checkpoint, and standalone non-LoRA MTP training parameters. For target-model evaluation, keep speculative MTP decoding disabled.

Download and evaluate

hf download LSW142857/OPSD-Qwen3.5-9B-LoRA-Medium-iter31 \
  adapter/checkpoint_iter_0000031.safetensors \
  adapter/checkpoint_iter_0000031.manifest.json \
  --repo-type model \
  --local-dir .

git clone --branch opsd-pi-training \
  https://github.com/LeiLiLab/privilege-code-opsd.git
cd privilege-code-opsd

QWEN_LORA_RANK=64 QWEN_LORA_ALPHA=128 \
bash examples/coding_agent_rl/evaluate_qwen35_9b_opsd_lora.sh \
  /path/to/checkpoint_iter_0000031.safetensors \
  /path/to/Qwen3.5-9B \
  /path/to/eval.jsonl

OPSD configuration

The frozen Base teacher received Medium privileged information separately for EXPLORE, REPRODUCE, DIAGNOSE, EDIT, and VERIFY. The student did not receive PI. SUBMIT executed but its tokens were masked. The objective used Top-64 conditional forward KL at temperature 0.6 plus one MTP layer at scale 0.05. LoRA learning rate was 1e-5; actor topology was TP8/DP1 and rollout topology was two TP4 engines.

The final update metrics were finite:

See config/training_lineage.json for machine-readable provenance and the two dataset SHA-256 values. Training code for the final continuation phase was commit 8e75a5500c751e755c20d4697200fcf1e53d0a7f.