SFT-4B-ScaleCUA-Qwen38

SFT-4B-ScaleCUA-Qwen38

Qwen/Qwen3-VL-4B-Instruct supervised-finetuned on single-teacher lite.scalecua trajectories (Qwen3.8-27B only). Final checkpoint, epoch 3 / 918 steps.

Serves as the init and reference model for HaoranLiu/DPO-ScaleCUA-CrossSource.

Training data

Hyperparameters

Results

lite.osworld eval, 332 tasks (369 minus exclude_reason), greedy, max_steps 30, group_size 1, mean over all 332. Every row measured under one identical protocol.

Three findings from the surrounding ablation

Multi-source beats single-source, but inside the grey zone. Balanced 3-source scores 118 vs this arm's 114 (+4 / +0.0106). The judging rule in use — a difference counts only at >0.02 mean and ≥7 tasks — leaves this undecided.

Source balance matters more than coverage. The balanced and priority-fill arms share an identical task set and identical data volume (1430 each); they differ only in how the source is distributed (33/33/33 vs 86/12/3). Balanced wins 118 vs 114. Meanwhile the priority-fill arm ties this single-source arm (114 vs 114) despite +206 tasks and +16.8% data — so the gain tracks diversity, not coverage.

Shortening the history window costs real accuracy. history=1 drops to 102 (−12 tasks / −0.0431, both past threshold — the only firm difference in the set). It buys a lot though: 6.0 GB → 989 MB of training data, 65 → 37 GB VRAM, 16.2 → 7.9 h.

Caveats

• Single seed, one sample per task at temperature 0. Repeated rollouts of one checkpoint move the mean by ~0.0035; re-running an identical checkpoint has differed by 2 tasks.
• The comparison against SFT-4B-LiteOS-MixedOpen changes two variables at once (dataset ScaleCUA vs LiteOS, and single- vs multi-source), so it is not a one-variable contrast. Only the ScaleCUA arms are strictly comparable to each other.