Downloads · 30 days
9
69% of all-time downloads
DPO-4B-MobileGym-fork
HaoranLiu/DPO-4B-MobileGym-fork
DPO-4B-MobileGym-fork is a machine learning model from HaoranLiu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Trajectory-level DPO on fork-point pairs, from HaoranLiu/SFT-4B-MobileGym. Epoch-2 checkpoint, 24 steps.
All-time downloads
13
Public
Parameters
4.4B
8.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors8.9 GB · 100%
At a glance
- License
- other
- Model type
- qwen3_vl
- Access
- Public
- Created
- Aug 16, 2026
- Updated
- Aug 16, 2026
- SHA
- 8792447a
Base models
Model card
From the Hugging Face model README
DPO-4B-MobileGym-fork
Trajectory-level DPO on fork-point pairs, from HaoranLiu/SFT-4B-MobileGym.
Epoch-2 checkpoint, 24 steps.
Task-id pairing yields zero pairs on this source (160 tasks, 1 sample each), so every pair comes from forking a failed trajectory at the judged first mistake.
- data:
HaoranLiu/DPO-fork-Qwen3-MobileGym, 51 of 58 pairs (7 with a quality-tagged chosen side dropped) - chosen:
gpt-5.5fork continuation · rejected:Qwen/Qwen3-VL-4B-Instruct, return 0.0
Hyperparameters
| objective | trajectory-level DPO, summed per-action log-probs (no length normalization) |
| dpo_beta | 0.1 |
| kl_coef | 0.0 |
| optimizer | Adam, beta1 0.9, beta2 0.95, eps 1e-8 |
| lr | 5e-7, cosine decay to 0, warmup fraction 0.1 |
| weight_decay | 0.1 |
| clip_grad | 1.0 |
| global_batch_size | 4 pairs/step |
| micro_batch_size | 1 |
| seq_length | 4096 |
| precision | bf16 |
| seed | 1234 |
| parallelism | TP=2, PP=1, CP=1, DP=1 (2xH100), sequence_parallel |
| optimizer sharding | distributed (ZeRO-1) + CPU offload, precision-aware |
| recompute | full, uniform, num_layers 1 |
| loss / advantage hooks | lite.train.dpo.dpo_loss_function / lite.train.dpo.compute_dpo_pair_weights |
Results
mobilegym eval, all 256 tasks, greedy, max_steps 45, group_size 1.
| model | steps | mean | success | success_rate |
|---|---|---|---|---|
| Qwen3-VL-4B-Instruct (base) | — | 0.2140 | 36 | 14.06% |
SFT-4B-MobileGym iter_129 | — | 0.3120 | 57 | 22.27% |
| SFT-4B-MobileGym (init) | — | 0.3656 | 67 | 26.17% |
| fork ep1 | 12 | 0.3704 | 68 | 26.56% |
| this ckpt — fork ep2 | 24 | 0.3757 | 70 | 27.34% |
| fork ep3 | 36 | 0.3605 | 66 | 25.78% |
| naive pairs ep1 (276 pairs) | 69 | 0.3758 | 72 | 28.12% |
| naive pairs ep2 | 138 | 0.3865 | 71 | 27.73% |
| naive pairs ep3 | 207 | 0.3829 | 71 | 27.73% |
- Type
- qwen3_vl
- License
- other
- Created
- Aug 16, 2026
- Updated
- Aug 16, 2026