Physical_AI_SO101_Cup_Nesting_ACT_Policy

DecisionFacts Physical AI Policy — SO-101 Cup Nesting (ACT)

An Action Chunking Transformer (ACT) manipulation policy trained on real-world teleoperation data from the SO-101 robotic arm (so_follower) for a cup nesting task: picking up a cup and seating it inside a second cup.

This is the policy counterpart to the DecisionFacts Physical AI SO-101 Cup Nesting dataset. It is released as a reference checkpoint — a demonstration of what a standard ACT recipe achieves on a single clean tranche of DecisionFacts teleoperation data, and a starting point for fine-tuning, benchmarking, and reproduction.

---

Model Summary

Why ACT. ACT predicts a chunk of future actions from a single observation rather than one step at a time. For contact-rich, precision-alignment tasks like nesting, this suppresses the compounding error and jittery re-planning that plague per-step behavioral cloning, and it produces smooth, committed trajectories at 30 Hz without a separate motion planner.

---

Training Data

Trained on the DecisionFacts SO-101 Cup Nesting teleoperation tranche, recorded in LeRobotDataset v3.0 format.

Every episode is an expert human teleoperation demonstration, captured with synchronized dual-camera RGB video and full proprioceptive state at 30 fps.

Quality control. Prior to training, all 200 episodes were screened for completeness, timing alignment, and coherent task execution. Eight episodes were removed for camera-timestamp defects — frame timestamps that drifted out of lock-step with the joint-state stream, which would inject misaligned vision/action pairs into training. The remaining 192 episodes were verified clean and used in full.

Domain randomization. Cup poses, positions, orientations, and workspace configuration were varied across episodes rather than repeating a fixed setup, so the policy sees a broader distribution of visual and spatial conditions than a single staged scene would provide.

Note on the public dataset repository. The linked dataset repo is a public evaluation sample of the DecisionFacts catalog and contains a subset of the episodes. This checkpoint was trained on the full 192-episode verified tranche. Exact reproduction of this checkpoint requires the full tranche — contact [email protected].

---

Training Procedure

Normalization statistics were taken from the dataset's meta/stats.json. The run completed end to end with no restarts, NaN losses, or dataloader failures — the eight defective episodes were the sole source of instability observed in earlier attempts, and removing them was sufficient to make the run clean.

<details> <summary><b>Reproducing the training run</b></summary>

pip install lerobot

lerobot-train \
  --policy.type=act \
  --dataset.repo_id=DecisionFacts/Physical_AI_SO101_Cup_Nesting_Task \
  --steps=100000 \
  --batch_size=8 \
  --output_dir=outputs/train/so101_cup_nesting_act \
  --job_name=so101_cup_nesting_act \
  --policy.device=cuda \
  --wandb.enable=true

Hyperparameters not listed above follow LeRobot's default ACT configuration (chunk size, optimizer, learning rate, backbone). Verify against config.json in this repository, which is authoritative for this checkpoint.

</details>

---

Usage

Load the policy

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained(
    "DecisionFacts/Physical_AI_SO101_Cup_Nesting_ACT_Policy"
)
policy.eval()
policy.to("cuda")

Run on a physical SO-101

lerobot-record \
  --robot.type=so101_follower \
  --robot.port=/dev/ttyACM0 \
  --robot.id=my_so101 \
  --robot.cameras="{ cam_front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, cam_top: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30} }" \
  --dataset.repo_id=<your-hf-user>/eval_so101_cup_nesting \
  --dataset.single_task="Nest the cup" \
  --policy.path=DecisionFacts/Physical_AI_SO101_Cup_Nesting_ACT_Policy

Camera naming matters. The policy expects observation keys observation.images.cam_front and observation.images.cam_top. If your camera keys differ, remap them before inference — silently swapping the two viewpoints will degrade performance badly rather than fail loudly.

Fine-tune on your own data

lerobot-train \
  --policy.path=DecisionFacts/Physical_AI_SO101_Cup_Nesting_ACT_Policy \
  --dataset.repo_id=<your-hf-user>/<your-dataset> \
  --steps=20000 \
  --output_dir=outputs/train/cup_nesting_finetune

---

Evaluation

Rollout success rates on physical hardware are not yet published for this checkpoint. We are running a standardized evaluation protocol (fixed trial count, randomized cup placements, held-out starting configurations) and will update this section with results, including per-stage breakdown (reach / grasp / align / nest) and failure-mode analysis.

Until then, treat this checkpoint as a reference training artifact, not a performance-validated product. If you evaluate it, we would like to hear what you find — open a discussion in the Community tab.

---

Intended Use

Intended for:

• Reproducing and benchmarking ACT training on real SO-101 teleoperation data
• A fine-tuning starting point for related tabletop manipulation and nesting/stacking tasks
• Baseline comparison against VLA, diffusion-policy, and world-model approaches on the same data
• Research and education in imitation learning and embodied AI

Not intended for:

• Deployment in production, commercial, safety-critical, or unsupervised settings
• Any embodiment other than the SO-101 follower arm without retraining or adaptation
• Operation around people, animals, or valuable objects without supervision and hardware limits

---

Limitations and Risks

• Single embodiment. Trained exclusively on SO-101. Kinematics, calibration offsets, and gripper behavior differ across arms; expect degradation even on another SO-101 without recalibration.
• Single task, narrow distribution. The policy learned cup nesting under one lab's lighting, backgrounds, camera mounts, and cup geometry. Novel cups, cluttered scenes, or different camera placement are out of distribution.
• Viewpoint-sensitive. Two fixed camera poses are baked into the learned representation. Moving a camera meaningfully changes the input distribution.
• No language conditioning. This is a single-task policy, not an instruction-following VLA.
• Imitation ceiling. ACT reproduces demonstrated behavior. It has no explicit recovery mechanism for states outside the demonstration distribution and may stall or repeat a failing approach.
• Unvalidated success rate. See Evaluation above.

Safety. This policy commands a real robot arm. Always maintain a clear workspace, enforce torque and joint limits, keep an emergency stop within reach, and supervise every rollout.

---

License & Access

Released under CC BY-NC 4.0 — research and evaluation purposes only, consistent with the license of the underlying dataset.

Commercial use, redistribution, or deployment of this policy (or models derived from it) in commercial products requires a separate license. Because this checkpoint is derived from the DecisionFacts Physical AI dataset, dataset licensing terms flow through to the model weights.

Licensing the Full Catalog

This checkpoint is trained on a sample tranche of a larger, continuously growing dataset. The full catalog includes additional tasks, larger per-task episode counts, and can be scoped to specific manipulation skills, environments, or diversity requirements.

We offer:

• Pilot tranches (200–500 episodes) sized for evaluation and fine-tuning experiments
• Production tranches (1,000+ episodes) for training and post-training at scale
• Custom collection for task categories or embodiments not yet in the public catalog
• Custom policy training on your task, your embodiment, or your environment

To discuss access to the full catalog, custom collection, or commercial licensing, contact [email protected].

---

Citation

If you use this model in your work, please cite it as follows:

@misc{decisionfacts_so101_cup_nesting_act,
  title        = {DecisionFacts Physical AI Policy --- SO-101 Cup Nesting (ACT)},
  credits      = {Prabhu Raghav, Sreeram B Unni, Balamurugan Pandi, Sriram Gopalan},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/DecisionFacts/Physical_AI_SO101_Cup_Nesting_ACT_Policy}}
}

Please also cite the underlying dataset:

@misc{decisionfacts_teleops_dataset,
  credits      = {Prabhu Raghav, Sreeram B Unni, Balamurugan Pandi, Sriram Gopalan},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/DecisionFacts/Physical_AI_SO101_Cup_Nesting_Task}}
}

And the ACT method:

@inproceedings{zhao2023learning,
  title     = {Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware},
  author    = {Zhao, Tony Z. and Kumar, Vikash and Levine, Sergey and Finn, Chelsea},
  booktitle = {Robotics: Science and Systems (RSS)},
  year      = {2023}
}

---

<div align="center">

DecisionFacts Inc · Physical AI data and policies for real-world robot learning

Dataset · Organization · [email protected]

</div>