MolmoBot-Pi0-DROID-absjoint-reasoninit

MolmoBot-Pi0 — DROID absolute-joint, reasoning-init full fine-tune

MolmoBot-Pi0 (a PaliGemma-decoder VLA with a flow-matching action expert), fully fine-tuned on equad DROID with absolute joint-pose action targets, starting from a reasoning-initialized backbone.

Checkpoint: step 18000 — best held-out validation loss (val_action_loss = 0.0041).

---

What "reasoning-init" means

Standard full fine-tuning starts from the base VLA weights. Here we first bake a frozen reasoning LoRA into the PaliGemma decoder, then full-fine-tune from that point. The reasoning adapter is not kept as a separate module — its delta is folded directly into the base weights:

W  <-  W + (alpha / r) * (B @ A)      # applied in-place, per target module

with r = 64, alpha = 128 (scaling 2.0), folded across 207 modules. The adapter came from a reasoning-only training run on the same DROID data (Gemini-annotated chain-of-thought, PaliGemma prompt format).

So reasoning enters as a weight-space prior on the initialization, not as an auxiliary loss and not as a runtime module. After the fold the model is a plain flat checkpoint — inference is identical to any other pi0 checkpoint, with no reasoning code path and no extra parameters.

On resume the fold is deliberately skipped (the delta is already inside the saved weights), so it is applied exactly once, at step 0.

---

Action space

Absolute joint pose, 8-dim per step:

Action horizon 16. At execution the prediction is used directly as the target joint pose — no current-position offset (contrast with the joint_delta variant, where target = current_q + pred).

Normalization statistics are bundled in assets/droid_equad/norm_stats.json; arm stats follow the observation.state distribution, gripper from the action gripper dim.

---

Training setup

Checkpoint selection

Validation runs every 1000 steps on 8 fixed held-out batches (flow noise reseeded so the metric is deterministic and comparable across steps). Retention keeps the latest checkpoint (for resume) plus the 2 best by val_action_loss.

Early stopping

Training was supervised by a self-chaining job that, each time a training job ended, read the cumulative validation series and either resumed training (if val was still improving) or stopped. It resumed 3 times and stopped at step 20000 once no new best appeared for 2 consecutive validation points — leaving 10000 of the budgeted steps unspent.

---

Validation curve

val_action_loss, every 1000 steps:

Roughly an 8× reduction from the first validation point, with most of the gain by step ~6000 and a slow, noisy tail after ~11000.

---

Contents

model.safetensors                    # flat full model (7.0 GB), reasoning delta folded in
metadata.pt                          # global_step and training bookkeeping
assets/droid_equad/norm_stats.json   # state + action normalization statistics
assets/train_config.pkl              # serialized openpi TrainConfig

Optimizer state is not included (10.9 GB) — this checkpoint is for inference and evaluation, not for resuming training.

Related

• shrg7/MolmoBot-Pi0-DROID-absjoint — same data and action space, without reasoning-init (the control)
• shrg7/MolmoAct2-DROID-absjoint — MolmoAct2 on the same absolute-joint DROID targets