lerobot_so101_base_sim_pickplace

SO-101 Pick & Place - 50K Checkpoint

This is a 50,000-step checkpoint of an ACT (Action Chunking Transformer) policy trained for pick-and-place tasks using the SO-101 robot arm in MuJoCo simulation.

📊 Training Details

• Training Steps: 50,000
• Loss: 0.089 (down from 4.4 at start)
• Dataset: 105 human demonstrations (teleoperation)
• Policy Type: ACT (Action Chunking Transformer)
• Framework: LeRobot
• Simulator: MuJoCo
• Device: Apple Silicon (MPS)

🎯 Performance

Evaluation Results (3 episodes):

• Success Rate: 66.7% (2/3 episodes)
• Average Steps to Success: ~360 steps
• Task: Pick cube from table and place in bin

Successful Episodes:

• Episode 1: ✅ Success in 300 steps
• Episode 2: ❌ Timeout at 500 steps
• Episode 3: ✅ Success in 424 steps

🤖 Model Architecture

• Backbone: ResNet18 for visual encoding
• Input:
  • Robot state (12D): joint positions (6), cube position (3), cube-to-bin vector (3)
  • Front camera image (480x640 RGB)
• Output: 6D joint position commands
• Parameters: ~52M

📦 Dataset

Trained on davidlinjiahao/lerobot_batch_001:

• 105 teleoperation demonstrations
• Recorded using SO-101 leader arm
• Task: Pick and place cube into bin
• Video + state observations at 30 Hz

🚀 Usage

With LeRobot

from lerobot.policies.act.modeling_act import ACTPolicy
import torch

# Load checkpoint
policy = ACTPolicy.from_pretrained("davidlinjiahao/so101_50k_checkpoint")
policy.eval()
policy.to("cuda")  # or "mps" for Apple Silicon

# Get action
observation = {
    "observation.state": state_tensor,
    "observation.images.front": image_tensor
}

with torch.no_grad():
    action = policy.select_action(observation)

Evaluation in MuJoCo

mjpython scripts/deploy.py \
    --policy-path davidlinjiahao/so101_50k_checkpoint \
    --episodes 10 \
    --render \
    --hz 10

📈 Training Progress

This is a mid-training checkpoint. Training continues to 200K steps with checkpoints every 1,000 steps:

• ✅ 50K: Current checkpoint (66.7% success)
• 🎯 100K: Target evaluation point
• 🏁 200K: Final checkpoint (expected >90% success)

🔗 Related

• Dataset: davidlinjiahao/lerobot_batch_001
• Final Model: davidlinjiahao/so101_overnight (training in progress)
• Framework: LeRobot

📝 Citation

@misc{so101_50k_checkpoint,
  author = {David Lin},
  title = {SO-101 Pick & Place - 50K Checkpoint},
  year = {2025},
  publisher = {HuggingFace Hub},
  howpublished = {\url{https://huggingface.co/davidlinjiahao/so101_50k_checkpoint}}
}

📄 License

Apache 2.0