lingbot-va-ur3-bimanual

🤖 LingBot-VA: UR3 Bimanual Fine-Tuned Checkpoint (10k Steps)

This is a private fine-tuned checkpoint of the LingBot-VA causal world model, specifically adapted for a UR3 Bimanual robotic setup.

📝 About this Checkpoint

• Task: "Pick up the green ball using the right arm and pass it to the left arm."
• Base Model: robbyant/lingbot-va-base
• Training Duration: 10,000 steps (resumed from pre-trained foundation).
• Final Metrics:
  • Action Loss: ~0.0044 (High precision motor control).
  • Latent Loss: ~0.23 (Stable physical imagination).

🚀 How to Use

This repository contains the Transformer weights only. To deploy this model, you must use the LingBot-VA codebase and provide the original VAE and Text Encoder.

Installation & Deployment

For detailed deployment instructions on a fresh PC, please refer to the inference-deployment-guide.md found in the original repository.

1. Environment: Python 3.10, PyTorch 2.9.0.
2. Assembly: Download the base robbyant/lingbot-va-base weights.
3. Injection: Replace the base transformer weights with the ones provided here.
4. Inference: Run the following command:

   NGPU=1 CONFIG_NAME='ur3_i2va' bash script/run_launch_va_server_sync.sh
   

📐 Architecture

This model uses a Mixture-of-Transformers (MoT) architecture with 5.3 billion parameters. It interleaves video and action tokens in a shared latent space to perform closed-loop world modeling.

---

Created by Gemini CLI for JeffrinSam.