pi05-button-order-baseline-lora-gbs4-pgb1-gpu4-step5000

pi05-button-order-baseline-lora-gbs4-pgb1-gpu4-step5000

Trained locally for 5,000 optimizer updates on four NVIDIA RTX A6000s. Global batch4 ×5,000updates =20,000 training examples. The complete run targets6,250updates =25,000examples, matching the sample budget of the reference single-GPU batch1 ×25,000-update experiment. This 5k checkpoint is uploaded immediately at the user request while training continues. Held-out evaluation of this intermediate checkpoint is pending. This repository contains full EMA serving weights, normalization and history assets, inference source and exact installed dependency versions. No policy-quality evaluation is claimed for this intermediate upload. The full optimizer, non-EMA parameters, training RNG and sampler cursor are included under resume_state/, alongside the EMA serving weights in params/. This is a Cartesian8 LoRA adaptation of π0.5.

Input: base and wrist RGB images, current flange xyz + unit XYZW quaternion (positive qx chart), measured finger opening divided by0.08, and task text. Output:20 absolute target flange poses and gripper commands (0closed,1open). Numeric state/actions use train-only STD normalization. Current-state tokens use a separate train-q01/q99 bounded view, max128tokens. Task prompt: Press the buttons in the same order as shown in the demonstration.

History: Current observations only. No Writer, Status head, subgoal, or semantic-memory inputs are used.

Training: LoRA Gemma2B and300M; fresh Cartesian projections and optional history modules trainable; pretrained backbones and time MLPs frozen. AdamW, global batch4 (one/GPU), no accumulation,250-step warmup, cosine5e-5→5e-6 through step6,250. EMA0.999^4 =0.996005996001 preserves the reference per-sample smoothing timescale. Checkpoints saved at5,000 and6,250updates. Episode split41/5/5; accepted train/validation/test windows:{'train': 6351, 'validation': 899, 'test': 596}. The original40ms synchronization and100ms maximum-gap filters are retained.

Gripper command labels require saved full-rate Joy edges matched to server requests and clock bounds. Unknown targets are NaN/false on disk and masked in both flow conditioning and loss. Measured width is state only. Confirmed training gripper labels cover only part of one episode and are all closed; this corpus does not establish a learned open/close policy. Validation has no known gripper labels, so its gripper metric is null, not zero. Evaluation reports known-component offline errors, not robot success rates.

Load

Use Python3.10 and a CUDA12 GPU environment. Download this repository, then:

python -m pip install -r requirements.txt

from load_model import load, observe
policy = load()
# Arrays: RGB uint8 images, float32 state[8]. Resized to224 with aspect padding.
observe(policy, base_rgb, state)  # Append every observed frame for the history model.
result = policy.infer({
    "image": base_rgb, "wrist_image": wrist_rgb, "state": state,
    "prompt": 'Press the buttons in the same order as shown in the demonstration.',
})
actions = result["actions"]  # (20,8)

For the history model, call observe(policy, base_rgb, state) for every original observation frame, including the current one, and policy.reset() between episodes. It uses the checkpoint's frozen vision encoder and appends frame indices in order; the policy samples0 across the full prefix. observe is a no-op for the model without history. The offline evaluator in code/integrations/robomme/franka_release.py demonstrates this with cached features. This bundle does not include demonstration images or private raw workspace evidence. Robot control must use the same Cartesian frame and tool convention as the training data.