nano-vla-arm-3d

physicalai-bmi/nano-vla-arm-3d

An open-vocabulary vision-language-action policy that drives an articulated arm from a perspective (3D-camera) view, running fully in your browser. Same see-be-told-act task as nano-vla-arm, but the scene is rendered through a fixed perspective homography — a receding tabletop, foreshortened blocks — so the policy must read a camera-like image, not a flat schematic. Run it live at https://physicalai-bmi.org/research/vla.

• Vision: a 40×40×3 perspective render of a 2-link arm + red/green blocks on a receding table.
• Language (open-vocab): any instruction, embedded on-device by all-MiniLM-L6-v2 (384-d).
• Action: two joint deltas. Whole forward pass (3 conv + MLP) runs in plain JS.
• Params: 119,986.

Results (held-out, closed-loop)

Trained by behavior cloning on a Jacobian-IK expert with DART state-noise injection. The perspective view (baked 3×3 homography, stored in config.Hinv) is applied identically in training and in the browser, so train == live. A small reference policy on a rendered task; not a foundation model.

Files: model.safetensors, vla.web.json (float32 for in-browser, verified 2.3e-7 vs safetensors), metrics.json. CC-BY-4.0, Institute for Physical AI @ BMI.