act_picking_up_a_white_cube_v1.6

ACT Picking Up a White Cube — v1.6

An Action Chunking with Transformers (ACT) policy trained with Hugging Face LeRobot to perform a tabletop manipulation task: visually locate, approach, grasp, and lift a white cube.

The policy was trained from demonstrations collected with a 6-DoF follower robot and a front RGB camera. The dataset collection strategy deliberately varied the cube's position and orientation across the robot's reachable tabletop workspace.

<video src="/leninangelov/act_picking_up_a_white_cube_v1.6/resolve/main/assets/inference_demo.mp4" width="40%" controls playsinline></video>

Authors: Lenin A. Villanueva Q. and Thomy J. Villanueva Q.

Model Summary

The model is task-specific and was trained through offline imitation learning from human-generated robot demonstrations.

Task

The objective of the policy is to execute a complete cube-picking behavior from visual and proprioceptive observations.

At inference time, the model receives:

• one RGB image from the front camera;
• the current 6-dimensional robot state.

It outputs a sequence of 6-dimensional robot actions representing:

1. shoulder_pan.pos
2. shoulder_lift.pos
3. elbow_flex.pos
4. wrist_flex.pos
5. wrist_roll.pos
6. gripper.pos

The policy therefore learns the visuomotor mapping from the observed configuration of the cube and robot to the joint/gripper commands required to approach and grasp the cube.

Dataset

The model was trained on:

leninangelov/act-training-data-picking-up-the-white-cube-v5.0

Dataset statistics

Spatial design of the demonstrations

The demonstrations were not collected by repeatedly placing the cube at approximately the same location.

Instead, the reachable tabletop workspace was conceptually divided into six imaginary spatial regions. These regions were used as a collection protocol to encourage broader spatial coverage.

The collection strategy was designed around three principles:

• Spatial coverage: demonstrations were distributed across the six regions of the table rather than concentrated around a single pickup location.
• Approximate balance: the number of cases associated with the different regions was intentionally kept balanced so that one part of the workspace would not dominate the training distribution.
• Within-region variation: cube placement was varied around each conceptual region instead of using a single fixed coordinate.
• Orientation variation: the cube was presented at different orientations, adding variation independently of its position on the table.

The regions were collection aids rather than labels supplied to the ACT policy. The policy receives images and robot state; it is not explicitly told which region contains the cube.

This design was intended to reduce positional memorization and encourage the visual policy to use the observed cube location when selecting its trajectory.

Model Architecture

This model uses the ACT architecture implemented in LeRobot.

Inputs

Visual observation

observation.images.front
shape = [3, 480, 640]

A single RGB front-camera observation is processed by a ResNet-18 visual backbone initialized from ImageNet-1K V1 weights.

Robot state

observation.state
shape = [6]

The state contains the positions of the five arm joints plus the gripper.

Outputs

action
shape = [6]

Each action controls the same six robot dimensions represented in the state.

ACT configuration

The policy predicts an action chunk of 100 future robot actions from the current observation. With the dataset operating at 30 Hz, 100 control steps correspond nominally to approximately 3.33 seconds of action horizon.

Training

Published checkpoint

This repository represents the checkpoint saved after:

160,000 optimization steps

The surviving training log at the checkpoint reported approximately:

step:      160,000
epoch:     27.29
loss:      0.045
grad norm: 2.417
learning rate: 1.0e-5

Important note about the 200K configuration

The saved train_config.json contains:

steps: 200000

This represents the configured target of the resumed training run, not the checkpoint represented by this model version.

The v1.6 model itself corresponds to the 160,000-step checkpoint. The same training lineage was subsequently continued toward later checkpoints/model versions.

Optimization configuration

Image-augmentation transformations were present in the LeRobot configuration but were disabled for this run.

Training environment

Training was performed on a personal laptop equipped with an:

NVIDIA GeForce RTX 3070 Laptop GPU

The project was developed in a Windows/WSL-based LeRobot workflow. The surviving training/resume logs for this checkpoint lineage show the later training stages running from a Windows Conda environment with CUDA enabled.

This is therefore an example of training a real-robot visuomotor Transformer policy on consumer laptop hardware, rather than a datacenter GPU.

Training Time

The model was trained through several resumed sessions rather than one uninterrupted process.

The surviving logs allow the following approximate reconstruction:

The checkpoint at 115K encountered a disk-space error while saving optimizer state. Training was subsequently resumed from the previous valid checkpoint, causing part of the 110K–115K interval to be recomputed.

The recovered logs therefore account for approximately 23 h 49 min of active process time leading to the 160K checkpoint, including the repeated work caused by that interruption.

The first surviving training session started on April 4, 2026, and the 160K checkpoint was saved on April 5, 2026 at approximately 15:23, giving a calendar span of roughly 35 h 10 min including interruptions and inactive periods.

These numbers should be treated as a reconstruction from the original logs, rather than as a controlled GPU-training benchmark.

Inference Configuration

The policy configuration stored with the model provides the following inference characteristics:

The saved LeRobot processor configuration performs the equivalent of:

RGB image + robot state
        │
        ▼
batch construction
        │
        ▼
move tensors to CUDA
        │
        ▼
mean/std normalization
        │
        ▼
ACT policy
        │
        ▼
100-step action chunk
        │
        ▼
action unnormalization
        │
        ▼
move action output to CPU
        │
        ▼
robot execution

The exact historical lerobot-record command used for the physical inference experiment was not preserved in the material currently available, so it is intentionally not reconstructed here from assumptions.

Inference Demo

The following video shows the trained policy operating the physical robot during inference.

<!-- Upload the video to this repository as: assets/inference_demo.mp4 -->

<video src="/leninangelov/act_picking_up_a_white_cube_v1.6/resolve/main/assets/inference_demo.mp4" width="40%" controls playsinline></video>

The video is intended as a qualitative demonstration of the learned behavior. It should not be interpreted as a statistically controlled success-rate evaluation.

Evaluation

Physical inference was used to inspect the policy behavior on the real robotic setup.

This card does not report a numerical task success rate because a sufficiently reliable aggregate success/failure record for the v1.6 checkpoint could not be reconstructed from the surviving experiment logs.

The inference video above therefore serves as qualitative evidence of the physical behavior of this checkpoint.

For a quantitative evaluation, future reporting should ideally include:

• number of physical trials;
• successful pickups;
• failures to reach the cube;
• grasp failures;
• success rate by tabletop region;
• success rate by cube orientation;
• in-distribution versus held-out positions.

Intended Use

This model is intended for:

• robotics research and education;
• experiments with ACT and imitation learning;
• analysis of visuomotor policies trained from relatively small real-robot datasets;
• tabletop object-manipulation demonstrations;
• reproducibility experiments using Hugging Face LeRobot.

Limitations

The policy was trained for a narrowly defined manipulation task and environment.

Performance may degrade with changes such as:

• substantially different camera viewpoints;
• different table geometry;
• significant lighting changes;
• different cube appearance or dimensions;
• robot calibration changes;
• positions outside the demonstrated workspace;
• occlusion of the cube;
• changes to robot dynamics or control timing.

The spatially distributed data-collection strategy increases variation within the training set but does not make the policy a general-purpose object-manipulation model.

The model should not be used as an unsupervised controller in safety-critical environments.

Reproducibility Notes

The most important configuration values required to reproduce the training setup are stored with the model in the LeRobot configuration files.

For exact reproduction, use:

• the same training dataset;
• the same ACT architecture;
• batch size 8;
• AdamW with learning rate 1e-5;
• ResNet-18 ImageNet initialization;
• mean/std normalization;
• 100-step action chunks;
• seed 1000;
• the same 640 × 480 front-camera observation layout.

Because LeRobot APIs and configuration formats can evolve, reproducing the experiment with a newer LeRobot release may require adapting the command-line interface while preserving these model and optimization parameters.

Related Resources

Training dataset

leninangelov/act-training-data-picking-up-the-white-cube-v5.0

Architecture

Action Chunking with Transformers (ACT), introduced in:

Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware arXiv:2304.13705

Although ACT was originally introduced for fine-grained manipulation, this model applies the policy architecture to a single-arm tabletop cube-picking task using LeRobot.

Authors

Lenin A. Villanueva Q. Thomy J. Villanueva Q.

Citation

If this model or its associated dataset is useful in your work, the following citation can be used:

@misc{villanueva2026actwhitecube,
  author       = {Lenin A. Villanueva Q. and Thomy J. Villanueva Q.},
  title        = {ACT Picking Up a White Cube v1.6},
  year         = {2026},
  publisher    = {Hugging Face},
  note         = {LeRobot ACT policy for real-robot tabletop manipulation}
}

License

Released under the Apache License 2.0.