act_so101_8bin_wrist_front_chunk50

ACT · SO-101 · 8-bin pick-and-place · wrist + front · chunk 50

Action Chunking Transformer trained with LeRobot 0.6.0 on BrutalCaesar/phi_so101_8bin_v1. Two cameras: wrist (gripper module) + front (desk-level, Logitech Brio 101).

One of six runs in a chunk-size × camera-pair sweep ({50, 75, 100} × {wrist+top, wrist+front}).

🚨 The camera keys are swapped — read this before inference

The dataset was recorded with the wrist and top camera keys transposed. This model therefore expects the physical wrist camera under the key observation.images.top:

Wire the overhead camera to observation.images.top and the policy gets a close-up it has never seen in that slot. It will not error — it will just behave badly. The key name is wrong; the model is not.

Camera framing matters too: the mounts must sit where they sat during recording. A camera that drifted between recording and evaluation silently invalidates the policy.

Inputs / outputs

Normalization ships with the checkpoint as LeRobot 0.6.0 processor files (policy_preprocessor*, policy_postprocessor*) — not baked into the policy. Load through ACTPolicy.from_pretrained so they are picked up; hand-rolling a forward pass without them produces actions in the wrong units.

Training

Everything except chunk_size and the camera pair is LeRobot default, deliberately — the sweep is meant to be comparable across people training the same task on the same defaults.

Train / held-out split

Trained on 89 episodes covering 6 of the 8 bins. Left bin 3 and right bin 2 are held out entirely — no episode touching them appears in training. Those two bins are the generalization test; the other six measure fit.

Evaluation

Not yet evaluated on hardware. No success rate is reported here because none has been measured. The planned protocol is 20 scored rollouts per bin, reported per bin and never averaged — an average over 8 bins hides exactly the failure this split is designed to expose.

Train loss of 0.055 says the model fits the demonstrations. It says nothing about whether it completes the task.

Usage

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("BrutalCaesar/act_so101_8bin_wrist_front_chunk50")

Or point a LeRobot eval script at it with --policy.path=BrutalCaesar/act_so101_8bin_wrist_front_chunk50, remembering the camera-key mapping above when you write --robot.cameras.

Known limitations

• Single observation frame, so no velocity information; the policy infers motion from wrist-view context alone.
• The CVAE latent is effectively collapsed at kl_weight=10.0 on this dataset (KL ≈ 0 throughout training), so this behaves close to a deterministic chunk regressor. Not a defect — expected for ACT — but do not read the latent as a source of behavioral diversity.
• Trained on one lighting setup in one room. No domain randomization beyond LeRobot's default image augmentation.