SFT-4B-LiteOS-MixedOpen

SFT-4B-LiteOS-MixedOpen

Qwen/Qwen3-VL-4B-Instruct supervised-finetuned on lite.osworld trajectories produced entirely by open-source models — no GPT data. Final checkpoint (epoch 3, 912 steps).

Serves as the init / reference model for HaoranLiu/DPO-LiteOS-SourceAblation.

Training data

Hyperparameters

Results

lite.osworld eval, 332 tasks (369 minus exclude_reason), greedy, max_steps 30, group_size 1, mean over all 332.

Two things to note when reading this:

It beats the single-GPT-teacher reference with 38% less data. 0.3439 vs 0.3220, +5 tasks, from 1217 trajectories instead of 1967 — and none of them from GPT. A smaller training set can only depress this arm, so the direction is not a data-volume artifact.

The epoch curve is steep and still rising. 0.2043 -> 0.3213 -> 0.3439 (+0.117, then +0.023). Epoch 1 is far below what the reference reaches, and it takes until epoch 3 to pass it. Training loss meanwhile falls 0.221 -> 0.144 -> 0.095, i.e. the model is memorising while eval is still improving. Open-source trajectories are longer (11.62 steps mean vs GPT's 8.36) and more meandering, so more passes are needed to extract the same competence. 3 epochs may not be the ceiling; it was capped to leave budget for DPO.

Caveats

• Single seed, one sample per task at temperature 0. Repeated rollouts of one checkpoint move the mean by ~0.0035.
• train.synth and the eval split are similar in construction, so some of the gain may be memorisation rather than capability. No base-model number was measured on this eval, so it is not established how much of the absolute score SFT contributes.
• The reference (SFT-4B-LiteOS) used a different cohort mix (perturb+synth vs synth-only), so it is not a strict one-variable comparison. A matched GPT-only arm on this exact pool was not run.