Downloads · 30 days
17
35% of all-time downloads
ASethi04/MolmoAct2-BimanualYAM-dual-lidar-umi-12k
MolmoAct2-BimanualYAM-dual-lidar-umi-12k is a robotics model from ASethi04. Use it for the robotics task on the model card, and read the license before you ship it in a product. It is set up for lerobot. The card lists the license as apache-2.0.
Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/dual-lidar-umi (115 episodes, 110,269 frames, 30 fps — dual handheld UMI grippers, LiDAR-tracked EE poses, two 600×800 fisheye wrist cameras, umi1 = left).
Downloads · 30 days
17
35% of all-time downloads
All-time downloads
49
Public
Parameters
5.6B
11.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors11.5 GB · 100%
How the weights are stored.
BF165.4B · 97%
From the Hugging Face model README
Fine-tune of allenai/MolmoAct2-BimanualYAM on brandonyang/dual-lidar-umi (115 episodes, 110,269 frames, 30 fps — dual handheld UMI grippers, LiDAR-tracked EE poses, two 600×800 fisheye wrist cameras, umi1 = left).
Action space: 14-D ABSOLUTE end-effector pose (converted from the dataset's per-frame
deltas): [umi1 xyz + rotvec, umi1_gripper, umi2 xyz + rotvec, umi2_gripper], each arm
SE(3)-re-anchored to its episode-start pose (frame 0 = identity; episode starts match YAM
home positions). action[t] = state[t+1] (next-frame absolute target), chunk 30 @ 30 Hz.
Grippers = width_mm / 125 in [0,1] (open ≈ 1.0), passed raw (not quantile-normalized).
Cross-arm transform is NOT observable in state — relative geometry comes from vision.
Trained with the lerobot PR#4249 fixes; inference_action_mode=continuous baked in.
Normalization stats are TRUE GLOBAL q01/q99 (upstream aggregation bias #4156 avoided).
| Steps / epochs | 12,000 / ~7.0 |
| Global batch | 64 (8 GPUs × 8) |
| VLM | LoRA r=64 @ 5e-5; action expert full FT @ 1e-4 |
| Schedule | cosine, 600 warmup, decay ratio 0.1 |
| Augmentation | brightness/contrast/saturation 0.8–1.2, hue ±0.02, sharpness 0.5–1.5, affine ±5°/5% |
| Cameras | image_keys=[umi1, umi2] — two fisheye wrist views only |
| Prompt strings | setup "bimanual umi grippers in molmoact2"; control "absolute end-effector pose" |
| Split / seed | 100/0 / 1000 |
norm_tag.Put all oranges in the bowl.Single task, 115 demos, no held-out val. Delta→absolute conversion assumes rotation components are axis-angle (verified empirically on all episodes). Validate with a no-motion action probe before arming.