lingbot-vla-v2-unibot-lora

LingBot-VLA 2.0 — UniBot LoRA (full G1_Dex1)

LoRA fine-tune of LingBot-VLA 2.0 (6.3B: Qwen3-VL-4B-Instruct backbone + Qwen2 action expert with MoE layers) on the UniBot G1_Dex1 dataset, merged into the base weights as a single full-model safetensors checkpoint.

• LoRA: rank 64, alpha 32, on q_proj/v_proj/o_proj (65M trainable / 6.3B total)
• Dataset: UniBot G1_Dex1 full — 32 tasks, 33,276 episodes
• Training: 10K steps, bf16 model + fp32 LoRA, single RTX 4090; final val/loss 0.085
• Control space: joint — arms (14), grippers (2), waist (4), base (3) in the LingBot-VLA canonical space; arm actions are state-relative deltas (subtract_state), meanstd normalization

Repo layout

checkpoints/global_step_step=10000/model/model.safetensors  # merged full-model weights (bf16)
lingbotvla_cli.yaml                                         # training config (paths sanitized)
norm_stats/unibot_full.json                                 # meanstd normalization stats

The nested checkpoints/.../model/ layout is intentional: the deploy code resolves lingbotvla_cli.yaml at Path(model_path).parent.parent.parent.

Usage

This checkpoint is loaded with the deploy path in the lingbot-vla-v2 codebase:

from deploy.lingbot_vla_v2_policy import LingbotVLAv2Server

server = LingbotVLAv2Server(
    path_to_pi_model="<this repo>/checkpoints/global_step_step=10000/model",
    robot_norm_path="<this repo>/norm_stats/unibot_full.json",
    use_length=50, chunk_ret=True, use_bf16=True,
)
server.reset("unibot/g1_dex1_full")

The tokenizer/processor is pulled from Qwen/Qwen3-VL-4B-Instruct (see tokenizer_path in lingbotvla_cli.yaml). The base model weights are already merged into model.safetensors — nothing else is needed.

A ready-to-use competition wrapper (UniBot V1 Challenge) lives at https://github.com/ezhoureal/unibot_submission.

Reproduction

The LoRA was trained and exported with https://github.com/ezhoureal/lingbot-vla-v2:

# train (see scripts/train_unibot_lora.py)
bash train.sh scripts/train_unibot_lora.py configs/vla/unibot/unibot_lora.yaml \
    --train.output_dir output/unibot_lora_full

# export LoRA-only Lightning ckpt -> this merged checkpoint
python scripts/export_unibot_lora_merged.py \
    --ckpt output/unibot_lora_full/checkpoints/global_step_step=10000.ckpt \
    --output-dir output/unibot_lora_full/checkpoints/global_step_step=10000/model

Open-loop validation on G1_Dex1_ArrangePlates: MSE 0.229 / MAE 0.176, ~0.35s per 50-step action chunk on RTX 4090 (bf16).