climbing-holds-rig-with-taxonomy

climbing-holds-rig-with-taxonomy

DP3-style point-cloud diffusion policy for grasping climbing holds with a Franka arm + LEAP hand, trained on the spring-testbed-era rig dataset (rlogh/climbing-holds-rig).

This is the taxonomy-conditioned model. The point cloud + robot state are fused with a 64-d grasp-type embedding (crimp / sloper / pinch / jug, one-hot) before being passed into the diffusion U-Net. At inference time the operator provides the grasp-type label.

The unconditioned counterpart (ablation) lives at rlogh/climbing-holds-rig-no-taxonomy.

Files

Training summary

• Dataset: 200 episodes (50 / grasp type), 24,621 valid samples
• Epochs: 3000 (cosine LR decay after 500-step warmup)
• Batch size: 128, AMP enabled
• Best loss: 0.001670 (final 0.001760)
• Wall time: 8.9 h on RTX 2080 Ti
• Conditioning: PointNet(1024×3) → 256-d + State MLP → 128-d + GraspType MLP → 64-d → Fuse → 512-d
• U-Net dims: (256, 512, 1024), 100-step cosine DDPM
• State / action dim: 23 (Franka 7 + LEAP 16)
• --no-grasp-conditioning flag: OFF (taxonomy conditioning ENABLED)

Usage (robot machine)

mkdir -p checkpoints/pc_with_taxonomy_rig
hf download rlogh/climbing-holds-rig-with-taxonomy \
    --local-dir checkpoints/pc_with_taxonomy_rig

python3 data_collection/evaluate.py \
    --checkpoint checkpoints/pc_with_taxonomy_rig/best.pt \
    --pull-dist 0.130 \
    --hold 1 --grasp-type crimp

evaluate.py autodetects the encoder type and conditioning state from the embedded config dict. --grasp-type is required and is fed into the model's grasp-type embedding branch.

Paired-evaluation result (spring testbed, 80 paired trials, 2026-05-15)

Primary metric: slip force from a parallel-spring linear ratchet testbed. Pull is a 13 cm impedance-controlled displacement (180°, toward robot base) after policy convergence. See repository README for full protocol.

Citation

Repository: github.com/rumilog/rock-climb