da3xvla-rtw-native-k160-100k

DA3-XVLA RoboTwin Native (K=160, frozen DA3, ckpt-100000)

DA3-XVLA architecture trained on RoboTwin 2.0 native dataset.

Training config

• Architecture: X-VLA (Florence-2 VLM + Depth Anything 3 backbone + Perceiver resampler + cross-attention fusion + DiT-style flow-matching action decoder)
• DA3 backbone: depth-anything/DA3-BASE, FROZEN
• Geometry resampler: single Perceiver, K=160 learnable queries, 6 cross-attention layers
• Action head: 30-step flow-matching action chunks, 20-dim action (2-arm × [xyz(3) + rot6d(6) + grip(1)])
• Input: NATIVE 224×224 (skip the 224→504 upscale via XVLA_DA3_NATIVE_INPUT=1)
• Cameras: head + left_wrist + right_wrist (3 views)
• Data: RoboTwin 2.0 meta_robotwin_all_3cam.json = 50 tasks × (50 clean + 500 randomized) = 27,500 trajectories
• Iters: 100,000 (4 GPUs × bs=16 = effective batch 64)
• LR: 1e-4 peak, cosine decay to 1e-5 (learning_coef=0.1 for VLM)
• Final loss: 0.0475 (last 500 steps median), below baseline 13279's 0.0524

Critical inference flags

export XVLA_DA3_NATIVE_INPUT=1       # MUST — model was trained with native 224 input
export XVLA_RGB_INPUT=1              # true RGB end-to-end (not BGR-as-RGB)
export XVLA_POSED_DA3=1              # posed multi-view DA3 (intrinsics + extrinsics)
export DXVLA_FAKE_PROPRIO=0          # use sim's real proprio (RoboTwin native)
export DXVLA_ACTION_SMOOTH=0.7       # EMA action smoothing (cross-chunk persistent)
export DXVLA_PRIMARY_CAM=head        # head_camera as primary view
export DXVLA_EPISODE_TIMEOUT=600     # 10-min per-episode hard cap (optional but recommended)

Action layout (20-dim, rotate6d)

left_xyz [0:3], left_rot6d [3:9], left_grip [9],
right_xyz [10:13], right_rot6d [13:19], right_grip [19]

Gripper convention: training-frame is "1=closed", so grip = 1.0 - sim_gripper (RoboTwin "1=open" → "1=closed").

Eval recipe (X-VLA paper-compatible)

• 50 RoboTwin tasks × 100 episodes (seeds 0..99 inside each task) × 2 configs (demo_clean, demo_randomized)
• head_camera as primary view (matches training)
• step_lim ≤ 200

Source code

The model architecture (DA3-XVLA) is custom — pulling the weights alone won't run. Source: https://github.com/2toINF (X-VLA fork) + customized DA3 integration.