flow_matching_carrot_in_pot_ee

Flow-Matching DiT · carrot-in-pot · EEF-delta (state 16 / action 7) — checkpoint 60k

Text-conditioned flow-matching policy (multi_task_dit, CLIP text encoder, Euler ODE) trained on the real UR7e "Put carrot in pot" demonstrations (54 GELLO-teleop takes, 30 fps) in the EEF-delta action space (eef_delta_v1). This is the 60k-step checkpoint (best open-loop chunk-30 error of 10k..100k; kanu kept only 100k).

Observation / action space (eef_delta_v1)

• observation.state 16-D = [q1..q6 (rad, UR order), tcp_x, tcp_y, tcp_z (m, base_link), r11, r21, r31, r12, r22, r32 (first two columns of the TCP rotation), grip_pos (0=open..1=closed)]. TCP = ur_kin.fk(q) + 0.174 m along flange +Z.
• action 7-D = [dx, dy, dz, drx, dry, drz, grip_cmd] — achieved per-frame TCP motion (dp = p_{t+1}-p_t, drot = so3_log(R_{t+1} R_t^T), base frame), gripper absolute 0..1. Deploy: p_target = p_live + dp, R_target = so3_exp(drot) R_live, analytic IK, branch locking.
• Cameras: cam1 (scene), cam2 (wrist), native 720×1280 fed in; the policy resizes internally to image_resize_shape = [224, 224] — do not pre-resize (double-resize silently degrades).
• Task string: "Put carrot in pot" (CLIP-conditioned; send it every tick).
• chunk_size 32, n_action_steps 24, n_obs_steps 2, num_integration_steps 100 at train time (10 is the usual serving override), DiT hidden 512 × 6 layers.

Training

• Dataset: carrot_in_pot_eef_lerobot_v3 — local LeRobot v3 re-export of Bigenlight/carrot_in_pot_lerobot_v3 (54 ep / 17,085 frames; joints de-lagged by per-take τ≈0.90 s + linear re-interpolation). Not yet on the Hub.
• lerobot-train, batch 8, seed 1000, 100k steps (10 h 15 m on one RTX A4000, kanu), job fm_carrot_eef.

Held-out results (open-loop, chunk-30 cumulative error, episodes 48–53)

lerobot's eval_loss rose monotonically 0.096→0.233 over the run while the open-loop metric kept improving — it was ignored for checkpoint selection.

Status

Real-robot closed-loop evaluation: not yet run. The shipped ZMQ servers (gello_policy/policy_server/fm_server.py) are joint-space 7/7 and refuse this checkpoint's 16-D state; serving needs the EEF-mode deploy node from branch feat/carrot-eef-il.